Network Anomaly Detection: Teaching a Network to Notice

An access point in a campus library had been fine for months. Then, over two weeks, it started rebooting a little more often — nothing dramatic, one extra reboot a day, then two. No threshold was breached, so no alert fired. On a Friday it failed completely during exams. A machine-learning model would have flagged that drift on day three — because it was watching the shape of the pattern, not a single line. That gap is what anomaly detection closes.

Anomaly detection is the sensing layer beneath everything else in AIOps — the part that decides what is worth reacting to in the first place.

Rather than re-argue why fixed thresholds fall short (we do that in AIOps vs traditional monitoring), this article goes the other way: how the machine actually learns "normal," the different shapes an anomaly can take, and why some of the most valuable ones are almost invisible.

Normal is a rhythm, not a line

The core idea is deceptively simple. Instead of a flat threshold, the engine learns a baseline for each metric — and a real network's baseline is never flat. It breathes: busy on weekday mornings, quiet at night, different again at weekends and holidays. A good model captures that rhythm, so a reading is only suspicious when it is unusual for that moment.

A threshold knows a number. Anomaly detection knows the story of that number — and notices when the story changes.

How the model learns "normal"

Under the bonnet, the engine is doing three things that a static threshold cannot. First, it models seasonality — the repeating daily and weekly patterns — so it expects the Monday-morning surge and does not panic when it arrives. Second, it tracks each metric per site and per device, because "normal" for an airport concourse is nothing like "normal" for a back-office switch; one global rule for both would be wrong for both. Third, it adapts: as legitimate usage shifts — a new tenant, a bigger intake of students, a seasonal rush — the baseline updates itself, so you are not endlessly re-tuning thresholds by hand as the business changes.

The trade-off is a short learning period. For the first days to a couple of weeks the model is still establishing what each metric's rhythm looks like, and it is deliberately cautious about firing during that window. Once the baseline is set, it becomes both more sensitive to genuine drift and quieter about routine variation — the opposite of a threshold, which is equally blunt on day one and day one thousand.

The three shapes an anomaly takes

Not every anomaly announces itself with a spike. Detecting all three shapes below is what separates a real model from a dressed-up threshold — and the third is where the campus-library story lived.

Point — the obvious one

A single reading lands far from anything normal: a port suddenly saturating, a device rebooting once, hard. Even a threshold catches most of these — the easy 10%.

Contextual — right value, wrong moment

The reading itself looks fine; it is the timing that is wrong. Heavy traffic at 3 a.m., a burst of authentications from an area that should be empty. A threshold has no clock and no map. A model that learned the rhythm sees it immediately.

Collective — the quiet killer

No single point is alarming, but the group is: a slow drift in reboots, a gently rising error rate, a fan creeping hotter week over week. This is the shape behind most "why didn't we see it coming?" outages — and the one anomaly detection is uniquely good at surfacing early.

What an early flag is worth

An anomaly on its own is just a smarter alert. Its value is what it enables next: correlation and root-cause analysis turn a cluster of anomalies into one diagnosed incident, and — when the fix is safe — a self-healing network can act on it automatically. Here is where that early flag pays off:

• The switch that ages out on schedule. Rising temperature and sporadic errors on one NetForce switch port drift from baseline for a week — long enough to swap the unit during a planned window.
• The quiet breach. A single laptop begins talking to dozens of internal hosts it never touched before. Against a learned baseline it is glaring; in a raw log it is invisible. Your security team gets a real lead.
• Capacity you can budget for. A steady multi-week climb at one branch is an anomaly against its own history — a costed reason to upgrade the uplink before users complain.

From a single flag to a resolved incident

A lone anomaly is rarely the whole story — and treating each one as its own alert is how you recreate the noise you were trying to escape. The value comes from what happens after detection. Related anomalies across different devices and metrics are grouped and passed to root-cause analysis, which asks: are these thirty flags thirty problems, or one? Almost always it is one — a single uplink, a single bad change, a single failing radio — and naming that one cause is what turns a wall of anomalies into a single, actionable incident. From there, if the fix is well understood and safe, a self-healing network can apply it and verify the result automatically. Detection is the first link in that chain; on its own it is useful, but joined to correlation and action it is transformative.

Which signals are worth watching, and where they come from

Anomaly detection is only as good as the signals feeding it. A model cannot find a pattern in a metric it never receives, and it cannot separate cause from symptom if it sees one layer of the network and not the others. Before evaluating any detection engine, work out what your estate can actually emit today.

Streaming device telemetry is the primary feed and the one that carries most of the useful signal. From NetWave access points: channel utilisation, noise floor, retry and drop rates, per-client SNR, association and roam counts, radio resets. From NetForce switches: per-port utilisation, CRC and FCS error counters, buffer and tail drops, PoE draw against budget, optic transmit and receive light levels, temperature. From NetGuard controllers: session and connection-table depth, CPU and memory, NAT translation counts, policy hit rates. Poll or stream at one to five minutes for capacity trends and at sub-minute resolution for anything with fast dynamics such as RF.

Syslog and structured events carry what counters cannot: authentication failures, DHCP pool exhaustion, spanning-tree topology changes, power supply and fan faults, licence and certificate expiry. These need parsing into structured records. A platform that only archives syslog gives you a search box, and a search box detects nothing.

SNMP remains necessary for the parts of the estate that will not stream: older switches, UPS units, environmental and door sensors, third-party equipment. It is coarser and heavier, so treat it as a way to close blind spots rather than as the main feed.

Flow records — NetFlow, sFlow or IPFIX — turn "this link is behaving abnormally" into "this link is behaving abnormally because a particular application or host changed its pattern". Without flow, a detected traffic anomaly is a question rather than an answer.

Two supporting inputs make an outsized difference. Configuration and change history lets the engine tie a shift in behaviour at 14:35 to a push at 14:32, instead of treating it as an unexplained anomaly. And client experience measurements — association time, DHCP time, DNS response time, time to first byte — are frequently the earliest place a problem shows, particularly on high-density campus wireless, where users notice degradation long before any device counter looks alarming.

Audit which of these your devices can produce before you shortlist a platform. The datasheets on our downloads page set out what Immunity hardware exposes, and for anything specific to your design you can ask us to confirm current capabilities in writing.

Where fixed rules still beat statistics

It would be neat to say that learned baselines replace thresholds entirely. They do not, and a platform that claims otherwise is worth questioning.

A fixed rule is a line drawn by a human: alert if a port is down, alert if a device is unreachable for sixty seconds, alert if a certificate expires within thirty days, alert if a security policy is violated. These conditions are binary, unambiguous and safety-critical. They have no meaningful "normal range", so there is nothing for a model to learn, and making them probabilistic would only add latency and doubt. Keep them as rules permanently.

A statistical baseline is the right instrument for everything with a normal range that varies by device and by hour: utilisation, error rates, client counts, latency, retry rates, memory growth, temperature. Here a fixed line is wrong almost by definition, because the same value is unremarkable on one device at one hour and alarming on another.

The practical differences worth holding in mind:

• Rules are instantly explainable; baselines are not, unless the platform makes them so. Insist on a view that shows the observed value, the learned envelope and the window used to build it. If a finding cannot be drawn that way, your team will not trust it and will not act on it.
• Rules do not scale with device diversity. A lecture-hall access point, a hostel access point and a warehouse access point need three different normals. Baselining learns all three; rules would need three policies and a maintenance burden that grows with every new site.
• Only baselines see drift. A threshold fires when a line is crossed. A baseline can flag a counter that has been creeping upward for three weeks and will cross that line next month, which is the entire basis of predictive maintenance.
• Rules are deterministic during change. During a migration or a major reconfiguration, learned models are temporarily unreliable because the "normal" they learned no longer applies. Rules keep working.

A sensible platform runs both and tells you which mechanism produced any given finding. The wider comparison is set out in AIOps vs traditional monitoring.

How long a baseline takes, and what to expect while it settles

The most common disappointment with anomaly detection is a timing problem rather than a technology problem: the platform is judged in week two, when it is still learning.

The learning period depends on the periodicity of the signal being modelled.

• Daily-cycle metrics — client counts, channel utilisation, WAN throughput, session counts — typically need one to two weeks before the engine can distinguish a genuine excursion from ordinary Tuesday-afternoon behaviour.
• Weekly-cycle metrics need four to six weeks, because the model needs to observe several weekends and several Mondays before a quiet Saturday stops looking like an outage.
• Seasonal patterns — academic terms, admission periods, festival seasons, financial-year close, quarterly stock counts — need a full cycle before the model has seen the pattern even once. Until then those periods will produce findings that a human must classify as expected.
• Sparse devices that see traffic only occasionally may never build a confident envelope of their own. Assign them to a role-based group so they inherit a sensible profile rather than being judged on a handful of samples.

What this means in practice is that the first two to four weeks after deployment will be noisier than steady state, and that this is expected behaviour rather than a defect. Plan for an engineer to spend half an hour a week reviewing findings and marking each as real, expected or noise. On most platforms that classification is precisely how the models improve, so the effort is not overhead; it is the training loop. Write the settling period into the project plan and set expectations with your management before go-live, not after the first noisy week.

Tuning false positives without going blind

Every detector trades false positives against false negatives. The question is not whether a platform produces false positives — all of them do — but whether it gives you honest controls over where that trade-off sits.

On a network, false positives usually come from four places. Genuine but uninteresting change, where a new floor or a new batch of users comes online and every metric legitimately shifts. Planned work, where a maintenance window is indistinguishable from a failure to a model that was never told about it. Sparse data, where too few samples make almost anything look anomalous. And correlated duplicates, where one root cause produces separate findings on every device downstream of it.

The levers that genuinely work, roughly in order of effect:

• Feed maintenance windows in from your change process. Suppression during planned work removes a large share of noise for very little effort, and it is usually the first integration worth building.
• Group devices by role rather than only by site. Every baseline in a well-formed group gets sharper, and sparse devices stop being judged on their own thin history.
• Raise the confidence floor for notification, not for detection. Keep detecting everything and keep it visible in the console; page a human only above a chosen confidence and severity. This is the key distinction — suppressing detection makes you blind, suppressing notification only makes you calmer.
• Require correlation before notification. Twenty findings from one uplink failure should arrive as one incident; this is the single largest reducer of perceived false positives, and the mechanics are covered in AI root-cause analysis.
• Use the feedback loop consistently. Marking a finding as expected should change future behaviour. Test this explicitly during a proof of concept: mark something, wait a week, and see whether it returns unchanged.

Hold the platform to a measurement rather than an impression. The most useful single number is the proportion of paged findings that led to any action at all; review it weekly and keep tuning. Do not accept a vendor's published detection accuracy as a substitute — the only figure that means anything is the one your own team produces after a month on your own traffic.

From detection to action, and the limits of automating it

A detected anomaly is not yet a resolved incident. What happens next falls into three tiers, and it is worth being clear about which tier a given finding belongs in.

Inform. The finding goes into the console for weekly review. Appropriate for low-confidence findings, slow drift with weeks of runway, and anything on a device the platform does not see well. Most findings should live here.

Page. A correlated, deduplicated incident with a named probable cause goes to an engineer. Appropriate for high-confidence findings with user impact, or a trend that will become an outage soon.

Act. The platform applies a defined remedy and verifies it. This tier is legitimate only for actions that are reversible, bounded to a small number of devices, and automatically verifiable within seconds or minutes — Wi-Fi channel and power re-optimisation being the standard example, as described in AI-driven RRM for Wi-Fi.

Anomalies must not be wired directly to actions on anything that fails those three tests. Routing or spanning-tree changes on core and distribution devices, firewall or segmentation policy changes, firmware upgrades, and reboots of any device that aggregates others should all produce a recommendation for a human, not an automatic change. A human must also stay in the loop during change freezes and wherever the platform holds low-confidence data.

Stated plainly, because language in this category is often loose: the scope of any automated remediation, the actions permitted on each device class, how each is verified and how each is rolled back must be agreed and recorded in a signed design document for your specific deployment before automation is enabled. Nothing here should be read as a commitment to high availability, automatic failover or guaranteed remediation on a particular network. Ask us to confirm current capabilities in writing for your scope; our engineers can be reached through the contact page. The boundaries are discussed further in self-healing networks explained.

A practical deployment sequence

Detection is easy to switch on and easy to abandon. A staged sequence keeps it useful.

Weeks 0 to 2 — instrument and verify. Onboard devices, confirm every feed is arriving, and close the blind spots: the switch nobody added, the access point behind a firewall rule that blocks telemetry, the syslog stream pointed nowhere, the flow exporter that was configured but never enabled. Detection may run, but nothing should page anybody. The only goal is data completeness.

Weeks 2 to 6 — settle and classify. Let the baselines build while a named engineer holds a short weekly review, classifying findings and feeding that back. Add maintenance-window suppression and role-based grouping as patterns become clear.

Weeks 6 to 10 — enable paging selectively. Switch on notification for the incident classes your team agrees matter, at one site or one building first, and confirm that a single root cause arrives as a single incident. Widen one site at a time. Resist recreating every rule from your previous system verbatim, which simply imports the old noise.

Week 10 onwards — consider narrow automation. Only after the above, and only for the safest action class, at a non-critical site, with full logging and a tested off switch. For multi-location estates, sequencing site by site is far easier to control than sequencing by device class — a point worth settling early when planning across an enterprise network.

What to measure to prove detection is working

Capture at least eight weeks of the following from your ticketing system and existing monitoring before the platform goes live, so there is something to compare against later:

• Share of incidents first reported by users rather than detected by the system. This is the cleanest single indicator that detection is doing something, and it is easy to count honestly.
• Median time to detect: the gap between the first symptom visible in the data and the first alert or ticket.
• Median time to restore, by category. Use medians; one long outage destroys the usefulness of a mean.
• Alert-to-action ratio — the proportion of notifications that produced any response.
• Repeat incidents: the same fault at the same site within thirty days, which tells you whether causes are being addressed or only symptoms.
• Predictive catches: findings raised before user impact, counted individually. Even a handful of documented cases makes a stronger argument internally than any headline percentage.
• Engineer hours spent on triage, estimated weekly by the team.

Re-measure the same list on the same definitions at ninety days and at six months. We deliberately publish no improvement percentages here, because a figure produced on another organisation's estate says nothing about yours — the outcome depends on your topology, your fault mix, your ticket hygiene and how disciplined the settling period was. Build the case from your own numbers; the method is set out in the AIOps ROI and business case article.

Common failure modes

• Blind spots in the estate. A third of devices never report, so the engine detects the symptom on the visible device and misses the cause on the invisible one.
• Judging the platform mid-settling. Evaluated in week two, declared noisy, abandoned in week three. The models had not finished learning.
• Suppressing detection instead of notification. Thresholds get raised to quieten the console, and the platform stops seeing the drift it was bought to find.
• No feedback discipline. Nobody classifies findings, so the models never improve and the noise never falls.
• Partial-stack visibility. An engine that sees wireless but not the wired path beneath it, or the wired path but not security policy, keeps producing confidently wrong root causes.
• Retraining through an incident. If the model keeps learning during a prolonged degradation, it can absorb the bad state as the new normal. Ask any vendor how they prevent this.
• No owner. Without a named owner the console becomes another browser tab nobody opens, regardless of how good the detection is.

All of these are avoidable with a staged rollout, honest expectations about the settling period and a named owner. If you are drawing up an evaluation, the questions to put to any vendor — including us — are collected in the AIOps platform buyer's guide, and we are happy to answer them in writing for NetCloud Central.

Frequently asked questions

Keep reading