EEG emotion recognition based on the TimesNet fusion model

In recent years, emotion recognition based on electroencephalogram (EEG) has become an important research field. This paper proposes an innovative multi-scale emotion recognition method (MS-ERM), which is based on a deep learning model. First, we divide the EEG signal into time windows of 0.5 s in different frequency bands to extract the differential entropy feature and embed the feature into the brain electrode map to express spatial information. Then, the features of each segment are used as input to the new deep learning model (MS-TimesNet). The model combines multi-scale convolution and TimesNet network to effectively extract dynamic time features, cross-channel spatial features, and complex time features in 2D space. Through extensive tests on the DEAP dataset, we prove that this method is superior to existing methods in terms of sentiment classification performance. In the arousal and valence classification, the average classification accuracy of subject-dependent tests reached 91.31% and 90.45%, respectively, while in subject-independent tests, the average classification accuracy was 86.66% and 85.40%, respectively. Code is available at this repository: https://github.com/hyao0827/MS-ERM. •An emotion recognition method MS-ERM based on deep learning is proposed. In this method, differential entropy (DE) is embedded in the electrode map to fuse spatial information as the input of the MS-TimesNet model.•The MS-TimesNet model is proposed, which combines multi-scale convolution and TimesNet network to extract complex spatio-temporal features at different levels.•Extensive tests on the DEAP dataset verify the superiority of MS-ERM, which is significantly higher than the existing methods.