Improved classification performance of EEG-fNIRS multimodal brain-computer interface based on multi-domain features and multi-level progressive learning

Abstract

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have potentially complementary characteristics that reflect the electrical and hemodynamic characteristics of neural responses, so EEG-fNIRS-based hybrid brain-computer interface (BCI) is the research hotspots in recent years. However, current studies lack a comprehensive systematic approach to properly fuse EEG and fNIRS data and exploit their complementary potential, which is critical for improving BCI performance. To address this issue, this study proposes a novel multimodal fusion framework based on multi-level progressive learning with multi-domain features. The framework consists of a multi-domain feature extraction process for EEG and fNIRS, a feature selection process based on atomic search optimization, and a multi-domain feature fusion process based on multi-level progressive machine learning. The proposed method was validated on EEG-fNIRS-based motor imagery (MI) and mental arithmetic (MA) tasks involving 29 subjects, and the experimental results show that multi-domain features provide better classification performance than single-domain features, and multi-modality provides better classification performance than single-modality. Furthermore, the experimental results and comparison with other methods demonstrated the effectiveness and superiority of the proposed method in EEG and fNIRS information fusion, it can achieve an average classification accuracy of 96.74% in the MI task and 98.42% in the MA task. Our proposed method may provide a general framework for future fusion processing of multimodal brain signals based on EEG-fNIRS.

Motor imagery electroencephalogram classification algorithm based on joint features in the spatial and frequency domains and instance transfer

“…Inspired by the above studies, in order to break through the limitations of single domain EEG features to varying degrees, provide more comprehensive and detailed EEG analysis, and to simultaneously reduce the impact of small sample sizes of MI-EEG samples from individual subjects and inter-individual differences across subjects on MI task classification models, this paper proposes an EEG joint feature classification algorithm based on instance transfer and ensemble learning. The algorithm includes three stages: the first stage is preprocessing of source and target domain data; the second stage is to use the CSP algorithm and Power Spectral Density (PSD) to extract spatial and frequency domain features, respectively, and to combine the two features as EEG joint features, so as to fully explore EEG and enrich EEG features, and improve MI identification performance (Bin et al, 2019;Qiu et al, 2022); The third stage is to classify MI-EEG using an ensemble learning algorithm based on kernel mean matching (KMM) (Xing et al, 2007) and adaptive enhancement of transfer learning (TrAdaBoost) (Dai et al, 2007). Firstly, the KMM algorithm is used to adjust the sample weights to make the feature distribution of the source domain closer to that of the target domain.…”