Experience with anomaly detection using ensemble models on streaming data at HIPA

Anomaly detection techniques are applied in many industry settings with great success. This paper presents experimental evidence of usefulness of these techniques in the High Intensity Proton Accelerator at the Paul Scherrer Institute. We present an anomaly detection model built as a combination of the Numenta NuPIC model and an LSTM autoregressor that works on streaming data from several beam diagnostic devices, automatically learning complex patterns from these signals after a relatively small adaptation time and small number of false positives. We show how this system could have alerted human experts of the failure of critical beam instrumentation tens of minutes in advance by finding anomalies preceding it, which stood unnoticed for several hours. We also present a full framework to exploit these models, which allows to monitor live data from the diagnostic devices, model results and diagnostic data and show historical data for easier model tuning.