Online Kernel Log Analysis for Robotics Application

Humanoid robot systems are composed of an assortment of hardware and software components, and they have complex embedded systems and real-time properties. These features make it difficult to isolate or to identify a fault in a short period of time even though such systems are expected to recover quickly in order to avoid any harmful behaviors that may cause harm to the users. This paper presents a new technological method for detecting errors in real-time applications online through the technique of online kernel log monitoring and analysis method. The contributions of approaches are that we present a method for kernel log analysis based on a state transition model of scheduling tasks, and apply it to the kernel logs to detect anomaly behavior of real-time tasks. In order to reduce the analysis overhead of huge volumes of data, we propose a new system that places the kernel log analysis engine on a separate core from the one that runs the kernel log monitoring process. Based on this system, we provide a framework for writing analyzers to detect errors incrementally. In our system, these components work together to solve the problems highlighted by root cause analysis in robotic systems. We applied the proposed system to actual robotics systems and successfully detected several deviated errors and faults that include a serious priority inversion that was not detected in over 10 years of operation in the actual operating system.