Driver’s facial expression recognition: A comprehensive survey

Driving is an integral part of daily life for millions of people worldwide, and it has a profound impact on road safety and human health. The emotional state of the driver, including feelings of anger, happiness, or fear, can significantly affect their ability to make safe driving decisions. Recognizing the facial expressions of drivers(DFER) has emerged as a promising technique for improving road safety and can provide valuable information about their emotions, This information can be used by intelligent transportation systems (ITS), like advanced driver assistance systems (ADAS) to take appropriate decision, such as alerting the driver or intervening in the driving process, to prevent the potential risks. This survey paper presents a comprehensive survey of recent studies that focus on the problem of recognizing the facial expression of driver recognition in the driving context from 2018 to March 2023. Specifically, we examine studies that address the recognition of the driver’s emotion using facial expressions and explore the challenges that exist in this field, such as illumination conditions, occlusion, and head poses. Our survey includes an analysis of different techniques and methods used to identify and categorize specific expressions or emotions of the driver. We begin by reviewing and comparing available datasets and summarizing state-of-the-art methods, including machine learning-based methods, deep learning-based methods, and hybrid methods. We also identify limitations and potential areas for improvement. Overall, our survey highlights the importance of recognizing driver facial expressions in improving road safety and provides valuable insights into recent developments and future research directions in this field. •A first comprehensive survey on recognizing facial expressions of drivers.•In-depth analysis of state-of-the-art methods for DFER.•Evaluation of the datasets used for recognizing the facial expressions of drivers.•Exploration of challenges and limitations in DFER field.•Discussion on potential future directions in DFER research.