A Multi-Source Fusion Approach for Driver Fatigue Detection Using Physiological Signals and Facial Image

Detecting driver fatigue is critical to ensuring road safety. Existing fatigue detection methods typically rely on traditional hand-picked features as inputs. However, these hand-picked features can hardly respond accurately to the driver's fatigue state due to a certain degree of subjectivity and the extraction of these features requires a long time window, which limits the accuracy and real-time performance of the detection. This paper proposes a novel fatigue detection method based on multi-source information fusion, which relies entirely on neural networks for automatic feature extraction. Through simulated driving experiments, we recorded physiological signals and facial videos from 21 participants for model training and testing. The results show that our model outperforms existing methods in terms of accuracy and real-time performance, achieving a detection accuracy of 93.15% within a 3-second time window (specificity = 94.04%, sensitivity = 91.71%). The visualization results of the model reveal potential relationships between facial regions for the first time, validating the rationality and effectiveness of our method. The practical issues of fatigue detection methods and future research directions are also explored.