Predictive Maintenance: Fixing Network Hardware Before It Fails

Network hardware rarely dies without warning. A switch that fails on a Friday afternoon was almost always running a little hotter, logging a few more errors, rebooting a little more often for days beforehand. Predictive maintenance is the part of AIOps that reads those quiet early signs and tells you which box is about to go — while you still have time to do something about it on your terms, not its.

It is the difference between a 2 a.m. emergency and a line item on next Tuesday's maintenance plan. Monitoring tells you a device has failed; predictive maintenance tells you a device is going to fail, and roughly when. That shift — from reacting to anticipating — is the same one that runs through anomaly detection, applied specifically to the health of the physical hardware.

Failure is a slope, not a cliff

The reason predictive maintenance is possible at all is that most hardware failures are gradual. A fan bearing wears and the fan spins faster to compensate, so temperature creeps up. A power supply ages and its rails drift. An optical transceiver's laser weakens and its light level falls, month over month. A memory leak eats free space until a process crashes. None of these crosses a red line on day one; each is a slope that ends, eventually, at a cliff. A fixed threshold only fires at the cliff edge. A model that has learned the device's normal behaviour sees the slope forming and raises a hand well before the edge.

How predictive maintenance works

Underneath, predictive maintenance runs four steps that turn raw device telemetry into a dated warning.

First it collects health telemetry from every device — not the traffic passing through, but the vital signs of the box itself: temperature, fan speed, power-rail voltages, CPU and memory use, error and CRC counters, reboot and crash logs, and, on fibre links, the optical light levels reported by the transceivers. Second, it learns a baseline of what healthy looks like for each specific device and component, because a "normal" temperature for a sealed outdoor unit is nothing like one for a rack switch in a cooled room. Third, it detects the drift — the slow, sustained departure from that baseline that signals wear rather than a momentary spike. Fourth, and most usefully, it estimates the remaining useful life: by extrapolating the trend, it turns "this is degrading" into "this is likely to fail in roughly a week," which is the number that lets you plan.

The machine learning that matters here is trend and survival analysis, not anomaly spotting alone. Spotting a single odd reading is easy; the hard, valuable part is recognising that a series of small, individually-unremarkable readings forms a trajectory heading somewhere bad, and estimating when it will arrive. That is a collective anomaly with a clock attached — and it is why predictive maintenance leans on the same baselining engine as the rest of AIOps, pointed at hardware health rather than user traffic.

From thresholds to a dated, confident forecast

The shift that makes prediction possible is from asking whether a value is over the line now to asking where it is heading and when it will arrive. A regression trend fitted to a metric's recent history is projected forward to the point it would cross the failure line — with better models fitting the accelerating curves real wear follows, not a straight line. Dating that crossing is remaining-useful-life estimation, and the borrowed technique is survival analysis: it asks how likely a component showing this pattern is to still be alive in three days, seven, or fourteen, learned from similar parts that failed before. The real lift comes from multivariate signal fusion — weighing temperature, fan speed, error counters and optical light together rather than alarming on each alone, so several signals drifting in a mutually-reinforcing way both sharpen the estimate and suppress the false alarms a lone spike would trigger. And it rests on a per-device baseline, because normal is local: a metric that climbs every weekday afternoon is following load, not wearing out, so the model learns each device's daily and weekly seasonality and reacts only to the residual drift once those cycles are subtracted. What reaches the team is specific — the device, the component, the trend, an estimated time-to-failure, and a confidence score that lets a near-certain warning be acted on tonight while a tentative one is watched another day.

The signals that precede a failure

Different components fail in different ways, and each announces itself through a characteristic signal that drifts before the break:

• Thermal — a slow rise in temperature, often because a fan is wearing out or airflow is blocked. Heat is the single most reliable predictor of hardware failure.
• Error counters — climbing CRC errors, input errors or discards on a port frequently precede a cable, optic or interface failure by days.
• Reboots and crashes — a device that used to run for months but has quietly started rebooting weekly is failing in slow motion, whether from a hardware fault or a memory leak.
• Power and fans — drifting power-supply rails, or a fan ramping to keep up, are early mechanical warnings that rarely reverse on their own.
• Optical light levels — on fibre links, the digital diagnostics from a transceiver report transmit and receive power. A steady fall in received light foreshadows a failing optic or a degrading fibre long before the link drops.
• Memory and resource leaks — free memory that trends downward toward zero over days will eventually crash a process; caught early, it is a scheduled reboot instead of an outage.

No single one of these is proof. The value comes from watching all of them, per device, against a learned normal — so a genuine trajectory stands out from the day-to-day noise, and a warning is only raised when the evidence actually points at wear.

Where predictive maintenance earns its keep

The value of a week's warning depends entirely on what fails and what it costs when it does. Predictive maintenance matters most where hardware is critical, hard to reach, or expensive to lose:

• Core and distribution switches — the boxes everything else depends on. A predicted failure on a core switch swapped in a maintenance window is a non-event; the same switch failing at peak takes a whole site with it.
• Fibre links and optics — degrading light levels on a transceiver are a textbook leading indicator. Catching the drift means replacing the optic on schedule rather than losing a backbone link without warning; the optics in a link are among the most predictable components to watch.
• Remote and outdoor sites — where an engineer visit is a planned expedition, not a walk down the corridor. Predicting the failure lets the trip carry the right spare and fix it once, instead of two emergency call-outs.
• PoE access points at height — ceiling- and pole-mounted units that are awkward to reach. Knowing which one is fading lets it be serviced during scheduled work rather than after users complain.
• Mission-critical environments — airports, hospitals and public networks where an unplanned outage is measured in stranded passengers or disrupted care, not just tickets. Here the ability to convert a failure into a scheduled swap is worth more than the hardware itself.

The intended effect, across all of them, is a change in how the network is run: from break-fix to plan-and-prevent. Unplanned outages fall because many failures are intercepted before they happen. Spare parts and engineer time are scheduled instead of scrambled, which lowers cost as well as stress. Truck rolls to remote sites drop because a visit is planned around a known fault with the right part in hand. And the hardware itself is used sensibly — replaced when it is genuinely wearing out, not on an arbitrary calendar or, worse, only after it has already taken a site down.

The switch that warned you

Take a distribution switch in a busy building. Over ten days its inlet temperature drifts up by a couple of degrees — nowhere near any alarm threshold — while its fan speed climbs to compensate and a handful of CRC errors begin appearing on one uplink. Under threshold monitoring, every one of those readings is still "green," so nothing fires; the first anyone knows is when the fan finally fails, the switch overheats and shuts down mid-afternoon, taking a floor of users with it and sending an engineer scrambling for a spare that may or may not be on the shelf.

Under predictive maintenance, the combination — rising temperature, climbing fan speed, new port errors, all drifting together against the switch's own learned baseline — is recognised on day three as the signature of a failing fan, and the platform estimates the unit has roughly a week left. A precise, evidence-backed warning goes to the team: this switch, this component, likely failure inside seven days. The spare is pulled from stock, the swap is booked into the next evening maintenance window, and the change takes ten quiet minutes with no users affected. Same fault, same hardware; the difference is entirely that it was seen coming. This is exactly the kind of drift a good anomaly-detection baseline surfaces, and once the cause is clear a self-healing network can even reroute traffic off the ailing switch in the meantime.

A second case: the optic that faded

The same logic catches a very different failure. On a fibre uplink carrying a building's backbone, each transceiver continuously reports its own diagnostics — the digital optical monitoring (DOM/DDM) figures for transmit and receive power. One receiver's light level begins to slip, a few tenths of a decibel a week, well inside the margin the link needs to stay error-free, so nothing drops and no alarm sounds; threshold monitoring sees a healthy link right until the received power falls below the sensitivity floor and a backbone segment goes dark without warning. Predictive maintenance reads the slope instead: tracking received power against the transceiver's own baseline, it projects the decline forward, estimates the optic will drop below the usable floor in roughly two weeks, and flags it with the evidence attached. Because the optics in a link are a supplied, field-replaceable part — Immunity trades NetBeam Optics rather than treating a link as a sealed unit — a replacement is swapped into a maintenance window long before the light level becomes a problem. A degrading optic is one of the most predictable components on the network, because it announces itself in a single, clean, monotonic number.

Where it falls short — and what it doesn't replace

Predictive maintenance is strongest where failure is gradual, and honest use means naming where that breaks down. The biggest limit is that not every failure is a slope: some components die instantly and silently — a capacitor shorts, a surge kills a board, a solder joint cracks — with no drift to read beforehand, so prediction shrinks the population of surprise failures without abolishing it. It also needs history to learn a baseline and calibrate a survival curve, so a newly deployed or rarely-failed device yields weaker estimates until that history builds up. False positives cost trust — every swap of a part that was actually fine burns time and teaches the team to distrust the next alert, which is why confidence scores and visible evidence matter. The method is also only as good as its telemetry: a device that never reports fan speed, or a link whose optical diagnostics go uncollected, is invisible to the forecast, so prediction leans on the same deep observability as the rest of AIOps. And it complements sensible lifecycle policy rather than replacing it — spares for critical parts, end-of-life retirement and designed-in redundancy still matter; forecasting makes those sharper, but a network with none of them is still one sudden failure away from an outage.

What good predictive maintenance looks like

Not every "health dashboard" is predictive. Four traits mark the real thing:

• It forecasts, not just reports. A good system estimates when, giving you lead time, rather than lighting up the moment a threshold is finally crossed.
• It learns per-device baselines. Normal differs by model, location and role; one global temperature rule is wrong for a rack switch and an outdoor AP alike.
• It fuses multiple signals. Temperature, errors, power and reboots together tell a story no single metric does; single-signal alerts are just thresholds again.
• It quantifies confidence and shows evidence. A warning you can act on names the component, the trend and the estimate — not a vague "device unhealthy."

Frequently asked questions

Keep reading