EEG spatial projection and an improved 3D CNN with channel spatiotemporal joint attention mechanism for emotion recognition

The EEG signals not only contain temporal information but also spatial information from the electrode positions in different brain regions. However, due to the low spatial resolution of EEG and individual variability, extracting both temporal and spatial information from multi-channel time-series EEG signals and constructing a corresponding feature extraction model is crucial for emotion decoding in EEG signals. Effectively extracting spatial and electrode position-related information from multi-channel time series EEG data is a critical and challenging aspect of EEG emotion decoding. Based on this, this paper proposes an emotion recognition algorithm based on EEG spatial tensor projection transformation and an improved 3D CNN model. According to the international 10–20 system, the preprocessed 32-channel EEG data is mapped to a 9 × 9 matrix corresponding to the electrode positions, Transforming multi-channel EEG data into a three-dimensional topological structure helps capture the spatio-temporal features of EEG signals. Considering the emotional response mechanisms of different brain regions and the multi-channel and spatial characteristics of EEG signals, we introduce a channel spatio-temporal joint attention mechanism and embed it into the deep convolutional network structure built with residual blocks, achieving an improved 3D CNN model. The deep convolutional network structure can explore finer, more subtle EEG signal features, enhancing the model's feature extraction capability. The channel spatio-temporal joint attention mechanism can explore emotional feature information from multi-channel EEG across different brain regions to highlight important EEG channels, as well as capture crucial spatial feature information in long-sequence signals. The method achieved an accuracy of 98.39% on the four-class emotion classification task in the DEAP dataset and 98.95% and 98.26% on the valence and arousal binary classification tasks, respectively. In the DREAMER dataset, the accuracy for the valence and arousal binary classification tasks reached 92.28% and 93.45%, respectively. We also conducted ablation studies to investigate the contribution of each module. The experimental result demonstrate that the EEG spatial–temporal projection transformation combined with the channel–spatial–temporal attention-based Res-3DCNN model can effectively extract emotional EEG features, achieving excellent EEG decoding performance.