Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Electroencephalography (EEG) signals elicited by multimodal stimuli can drive brain-computer interfaces (BCIs), and research has demonstrated that visual and auditory stimuli can be employed simultaneously to improve BCI performance. However, no studies have investigated the effect of multimodal sti...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Computers in biology and medicine 2021-09, Vol.136, p.104685-104685, Article 104685
1. Verfasser: Onishi, A.
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Electroencephalography (EEG) signals elicited by multimodal stimuli can drive brain-computer interfaces (BCIs), and research has demonstrated that visual and auditory stimuli can be employed simultaneously to improve BCI performance. However, no studies have investigated the effect of multimodal stimuli in rapid serial visual presentation (RSVP) BCIs. The present study proposed a rapid serial multimodal presentation (RSMP) BCI that incorporates artificial facial images and artificial voice stimuli. To clarify the effect of audiovisual stimuli on the RSMP BCI, scrambled images and masked sounds were applied instead of visual and auditory stimuli, respectively. The findings indicated that the audiovisual stimuli improved performance of the RSMP BCI, and that P300 at Pz contributed to classification accuracy. Online accuracy of the BCI reached 85.7 ± 11.5 %. Taken together, these findings may aid in the development of better gaze-independent BCI systems. •This study proposed a P300-based RSMP BCI that uses artificial face and voice stimuli.•Audiovisual stimuli enhanced classification accuracy of the RSMP BCI.•P300 at Pz contributed to the classification of the BCI.
ISSN:0010-4825
1879-0534
DOI:10.1016/j.compbiomed.2021.104685