A graph-based approach to detect unexplained sequences in a log

•A graph mining approach has been designed for recognizing anomalous sequences.•It supports both real time and batch processing for large scale data analysis.•A probabilistic penalty graph has been used for modeling log temporal sequences.•The approach’s effectiveness has been evaluated for different system configurations. In this paper we challenge the issue of detecting anomalous events in computer systems log files, through a novel graph mining approach. The basic idea is to model log temporal sequences as a particular graph and event detection as a particular path finding problem. Thus, anomalous sequences correspond to log parts that can not be “explained” by any path in the graph. We propose a novel Iterative Partitioning Log Mining technique to parse any kind of logs and to model their temporal sequence as a probabilistic penalty graph. The approach has been implemented in a framework supporting both real time and batch processing realized on the top of the Apache Spark analytics engine for large-scale data processing. Experimental results show the advantages of the proposed framework in terms of effectiveness for different system configurations.