Applying deep neural networks for multi-level classification of driver drowsiness using Vehicle-based measures

•Deep neural networks improve the accuracy of driver drowsiness detection.•Different levels of drowsiness are classified using deep convolutional networks.•Recurrent layers outperform convolutional networks for drowsiness detection.•Vehicle data are sufficient to have a good accuracy for drowsiness classification. Drowsy driving is one of the main causes of road accidents. Accurate and reliable detection of drivers' drowsiness is significantly important to prevent drowsiness-related accidents. In the context of automated vehicle driving, it is important for intelligent systems to know the current state of the driver to prepare handover maneuvers. Previous studies are mostly based on manually extracted features from either driving performance or driver physiological data. This methodology of a priori defined features can lead to losing valuable information of input signals that are significant to classify drowsiness levels in individual drivers because generally, it is not known which features are suitable for drowsiness prediction before classification. By using deep neural networks, features can be extracted automatically from preprocessed data. This paper presents a new non-obtrusive drowsiness detection system based on deep neural networks using vehicle-based measures. The proposed method is based on a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN). Five vehicle-based measures, including lateral deviation from road centerline, lateral acceleration, yaw rate, steering wheel angle, and steering wheel velocity, are exploited as network inputs. The level of drowsiness is classified into three different classes. Long-short term memory (LSTM) and gated recurrent unit (GRU) layers are used as RNN in the structure of the designed deep network. The performance of the proposed method is evaluated on experimental data that were collected from 44 sessions in a fixed-base driving simulator simulating monotonous night-time highway drives. Results show that the classification accuracy of the designed deep networks outperforms traditional classifiers like support vector machine and k-nearest neighbors. The highest accuracy of 96.0% has been achieved with a combination of CNN and LSTM (CNN-LSTM). Further research should include more signal sources, including unobtrusively taken physiological signals, and test the system in real-world conditions.