ML
Isolation Forest for sensor anomalies: how it works and where it misleads
Read flagged rows in a sensor table by their isolation score, understand what contamination does in the model, and see where the method misleads you.
SPC control charts and anomaly detection: same question, different language
Control charts have watched process drift for decades. Before proposing a machine learning model, look at what this simple method says.
In short
- A point beyond Shewhart limits is an inspection flag that the process may be outside its known variability, not a fault diagnosis.
- Control limits should be set from a period known to be stable, because limits set inside a drift make the drift look normal.
- CUSUM and EWMA charts are designed for small, persistent shifts that a Shewhart limit catches late.
What a control chart shows
A Shewhart chart draws a centre line and, typically, upper and lower control limits three standard deviations from it. A point outside the limits suggests the process may be outside its known variability. That is an inspection flag, not a fault diagnosis.
Which data to compute limits from
Control limits should be computed from a period in which you know the process was stable. If you compute limits from inside the drift, the drift becomes normal. The principle of separating training from evaluation applies here too: the data that sets the limit and the data you test on should be different.
Different tools for small shifts
A Shewhart limit catches small but persistent shifts in the mean late. Cumulative-sum (CUSUM) and exponentially weighted moving-average (EWMA) charts are designed for such small shifts. Autocorrelated data violates the assumptions of classic charts; monitoring residuals may then be necessary.
Compare against machine learning
Before proposing an anomaly model, build a robust control chart on the same data and record its false-alarm count. If the proposed model cannot beat this baseline in a way an engineer can understand, its complexity cannot be justified. This is a suggested evaluation approach; exact limit settings depend on the process and risk tolerance.
Frequently asked
How do I set control limits for a Shewhart chart?
A Shewhart chart draws a centre line and, typically, upper and lower limits three standard deviations from it. Compute the limits from a period when the process was known to be stable. Limits computed inside a drift make the drift look normal, so keep the data that sets the limit separate from the data you test on.
Should I use a control chart or machine learning for anomaly detection?
Build a robust control chart on the same data first and record its false-alarm count. If a proposed model cannot beat this baseline in a way an engineer can understand, its complexity is not justified. CUSUM and EWMA charts are designed for small, persistent shifts, and autocorrelated data may call for monitoring residuals.
Sources and further reading
Pilot steps in this guide are proposed engineering workflows. Examples do not constitute measured field performance.
Questions on this topic
Keep reading.
Full library
ML
Multivariate process monitoring with Hotelling T² and SPE
Read correlated sensors through T² and SPE together rather than one statistic, interpret contribution plots, and learn where the method’s assumptions break.
ML
Without fault labels: unsupervised machine learning in industry
Learn what unlabelled sensor history can and cannot tell you, how to evaluate a detector without labels, and why operating modes need attention first.