Two-dimensional phase lag index image representation of electroencephalography for automated recognition of driver fatigue using convolutional neural network

•Phase lag index method avoids the volume conduction problem.•Deep learning combines the two steps of feature extraction and classification.•The link of convolutional neural network and functional brain network is effective. Driving in fatigue state will increase the occurrence probability of related traffic accidents and cause severe economic and societal problems. To tackle the issue, a deep learning approach is proposed for the automated recognition of driver fatigue using electroencephalography (EEG) signals obtained from real driving. The methodology here proposed consists of converting the multi-channel EEG recording into functional brain network (FBN) adjacency matrices based on phase lag index (PLI) and feeding them into various convolutional neural networks (CNN) as input. These CNN models with convolutional layer, rectifier linear activation unit (ReLU), pooling layer and fully connected layer are designed to extract hidden features from images representing FBN adjacency matrices and then to achieve the two-ways classification task. The experimental results indicate that the highest classification accuracy of 95.4 ± 2.0%, highest sensitivity of 93.9 ± 3.1%, highest precision of 95.5 ± 2.4%, highest F1 score of 94.7 ± 2.0% and highest value of area under the receiver operating curve (AUC-ROC = 0.9953) are achieved using Model 4 based on PLI adjacency matrices as input with the 10-fold cross validation strategy. Indeed, all the CNN models considered in this research achieved accuracy higher than 94.40%. It is hence concluded that the proposed CNN models have the ability to self-learn and pick up more distinguishable features from the input data without a separate feature extraction or feature selection procedure. The experimental results also confirmed the effectiveness of the combination of FBN and CNN for the recognition of driver fatigue.