Visual analysis of fatigue in Industry 4.0

The performance of manufacturing operations relies heavily on the operators’ performance. When operators begin to exhibit signs of fatigue, both their individual performance and the overall performance of the manufacturing plant tend to decline. This research presents a methodology for analyzing fatigue in assembly operations, considering indicators such as the EAR (Eye Aspect Ratio) indicator, operator pose, and elapsed operating time. To facilitate the analysis, a dataset of assembly operations was generated and recorded from three different perspectives: frontal, lateral, and top views. The top view enables the analysis of the operator’s face and posture to identify hand positions. By labeling the actions in our dataset, we train a deep learning system to recognize the sequence of operator actions required to complete the operation. Additionally, we propose a model for determining the level of fatigue by processing multimodal information acquired from various sources, including eye blink rate, operator pose, and task duration during assembly operations.