Deep Neural Network with Joint Distribution Matching for Cross-Subject Motor Imagery Brain-Computer Interfaces

Abstract

Motor imagery brain-computer interfaces (BCIs) have demonstrated great potential and attract world-spread attentions. Due to the nonstationary character of the motor imagery signals, costly and boring calibration sessions must be proceeded before use. This prevents them from going into our realistic life. In this paper, the source subject's data are explored to perform calibration for target subjects. Model trained on source subjects is transferred to work for target subjects, in which the critical problem to handle is the distribution shift. It is found that the performance of classification would be bad when only the marginal distributions of source and target are made closer, since the discriminative directions of the source and target domains may still be much different. In order to solve the problem, our idea comes that joint distribution adaptation is indispensable. It makes the classifier trained in the source domain perform well in the target domain. Specifically, a measure for joint distribution discrepancy (JDD) between the source and target is proposed. Experiments demonstrate that it can align source and target data according to the class they belong to. It has a direct relationship with classification accuracy and works well for transferring. Secondly, a deep neural network with joint distribution matching for zero-training motor imagery BCI is proposed. It explores both marginal and joint distribution adaptation to alleviate distribution discrepancy across subjects and obtain effective and generalized features in an aligned common space. Visualizations of intermediate layers illustrate how and why the network works well. Experiments on the two datasets prove the effectiveness and strength compared to outstanding counterparts.

Improving the Cross-Subject Performance of the ERP-Based Brain–Computer Interface Using Rapid Serial Visual Presentation and Correlation Analysis Rank

“…RSVP systems ( Ratcliffe and Puthusserypady, 2020 ) were proposed to solve this problem, but the defect of these protocols was the long duration of the experiment time that deteriorates the ITR and lowers accuracy. Many scholars have done extensive research on improving the performance of BCI, such as creating a P300 speller performance predictor based on RSVP multifeature ( Won et al, 2019 ), finding the optimal features ( Yin et al, 2015 , 2016 ; Won et al, 2018 ), and using deep neural network algorithm ( Zhao et al, 2020 ), which cannot have a better result in a large subject study. In our study, the average ITR of the RSVP speller is 43.18 bits/min, which is 13% higher than the matrix paradigm, and we achieved the better accuracy results at present in a large subject study.…”

EEGG: An Analytic Brain-Computer Interface Algorithm

“…The most important detail was that subjects of this dataset were explicitly asked to imagine the kinesthetic experience rather than imagining the visual experience [26], [40]. Besides, an algorithm with high accuracy for a subject may perform terribly for other subjects, which is called a cross-subject problem [45]. Thus, this paper tested cross-subject accuracy.…”

EEGG: An analytic brain-computer interface algorithm

“…The most important detail was that subjects of this dataset were explicitly asked to imagine the kinesthetic experience rather than imagining the visual experience [32,37]. Besides, an algorithm with high accuracy for a subject could perform terribly for other subjects, which is called a cross-subject problem [39]. Thus, this paper tested cross-subject accuracy.…”