ELOF: fast and memory-efficient anomaly detection algorithm in data streams

Anomaly detection in multivariate data is an import research field. Many studies have been proposed aiming to develop the local outlier factor (LOF). However, the existing LOF-based models have two major problems: (1) need a large amount of memory to store data; (2) unsatisfactory detection results in high-dimensional data. To this end, we propose a new data streams anomaly detection algorithm extract local outlier factor (ELOF). To reduce data storage, we first design a memory window mechanism to limit the amount of data storage; then, we design a new sub-data extraction model to extract the sub-data of the original data information. Through these two designs, the amount of data storage can be effectively reduced. Moreover, the model framework is based on the density discriminant method, and it can be widely applied to different real scenarios without any prior information or assumptions of data distribution. The final comprehensive experimental results show that the ELOF model has a great improvement than many common models in terms of accuracy. Furthermore, the running time of ELOF algorithm is less than 1% of the original LOF algorithm under the same data set. These results indicate that the ELOF improved model consumes less memory in real-time data streams anomaly detection and works better in high-dimensional data streams detection.