A Fusion Framework for Confusion Analysis in Learning Based on EEG Signals

Human–computer interaction (HCI) plays a significant role in modern education, and emotion recognition is essential in the field of HCI. The potential of emotion recognition in education remains to be explored. Confusion is the primary cognitive emotion during learning and significantly affects student engagement. Recent studies show that electroencephalogram (EEG) signals, obtained through electrodes placed on the scalp, are valuable for studying brain activity and identifying emotions. In this paper, we propose a fusion framework for confusion analysis in learning based on EEG signals, combining feature extraction and temporal self-attention. This framework capitalizes on the strengths of traditional feature extraction and deep-learning techniques, integrating local time-frequency features and global representation capabilities. We acquire localized time-frequency features by partitioning EEG samples into time slices and extracting Power Spectral Density (PSD) features. We introduce the Transformer architecture to capture the comprehensive EEG characteristics and utilize a multi-head self-attention mechanism to extract the global dependencies among the time slices. Subsequently, we employ a classification module based on a fully connected layer to classify confusion emotions accurately. To assess the effectiveness of our method in the educational cognitive domain, we conduct thorough experiments on a public dataset CAL, designed for confusion analysis during the learning process. In both subject-dependent and subject-independent experiments, our method attained an accuracy/F1 score of 90.94%/0.94 and 66.08%/0.65 for the binary classification task and an accuracy/F1 score of 87.59%/0.87 and 41.28%/0.41 for the four-class classification task. It demonstrated superior performance and stronger generalization capabilities than traditional machine learning classifiers and end-to-end methods. The evidence demonstrates that our proposed framework is effective and feasible in recognizing cognitive emotions.