A novel multiple time-frequency sequential coding strategy for hybrid brain-computer interface

Background: For Brain-computer interface (BCI) communication, electroencephalography provides a preferable choice for its high temporal resolution and portability than other neural recording techniques. However, current BCIs could not sufficiently use the information from the time and frequency domain simultaneously. Thus, we proposed a novel hybrid time-frequency paradigm to investigate better ways of using the time and frequency information. Method: We adopt multiple omitted stimulus potential (OSP) and steady-state motion visual evoked potential (SSMVEP) to design the hybrid paradigm. A series of pre-experiments were implemented to study factors that would influence the feasibility of the hybrid paradigm and the interaction of multiple features. After that, a novel Multiple Time-Frequencies Sequential Coding (MTFSC) strategy was introduced and explored in experiments. Results: Omissions with multiple short and long duration could effectively elicit time and frequency features, including the multi-OSP, ERP, and SSVEP in this hybrid paradigm. The MTFSC is feasible and efficient. The preliminary online analysis showed that the accuracy and the ITR of nine-target stimulator over thirteen subjects were 89.04% and 36.37 bits/min. Significance: This study firstly combined the SSMVEP and multi-OSP in a hybrid paradigm to produce robust and abundant time features for coding BCI. Meanwhile, The MTFSC proved feasible and showed great potential in improving performance, such as expanding the number of BCI targets by better using time information in specific stimulation frequencies. This study holds promise for promoting better BCI systems design with a novel coding method.