Semi-supervised generative and discriminative adversarial learning for motor imagery-based brain–computer interface

Abstract

Convolutional neural networks (CNNs), which can recognize structural/configuration patterns in data with different architectures, have been studied for feature extraction. However, challenges remain regarding leveraging advanced deep learning methods in BCIs. We focus on problems of small-sized training samples and interpretability of the learned parameters and leverages a semi-supervised generative and discriminative learning framework that effectively utilizes synthesized samples with real samples to discover class-discriminative features. Our framework learns the distributional characteristics of EEG signals in an embedding space using a generative model. By using artificially generated and real EEG signals, our framework finds class-discriminative spatio-temporal feature representations that help to correctly discriminate input EEG signals. It is noteworthy that the framework facilitates the exploitation of real, unlabeled samples to better uncover the underlying patterns inherent in a user’s EEG signals. To validate our framework, we conducted experiments comparing our method with conventional linear models by utilizing variants of three existing CNN architectures as generator networks and measuring the performance on three public datasets. Our framework exhibited statistically significant improvements over the competing methods. We investigated the learned network via activation pattern maps and visualized generated artificial samples to empirically justify the stability and neurophysiological plausibility of our model.

Survey on the research direction of EEG-based signal processing

“… Accuracy comparison of different methods on the BCI competition IV dataset 2a. Each label on the vertical axis represents a method in a study, which is from Gaur et al ( 2021 ); Lashgari et al ( 2021 ); Lian et al ( 2021 ); Liu and Yang ( 2021 ); Liu et al ( 2021 ); Qi et al ( 2021 ); Ali et al ( 2022 ); Ayoobi and Sadeghian ( 2022 ); Chang et al ( 2022 ); Chen L. et al ( 2022 ); Ko et al ( 2022 ); Li and Sun ( 2022 ); Li H. et al ( 2022 ), and Tang et al ( 2022 ), from top to bottom, respectively. …”

“…A detailed table with information on the directions and performance of each paper can be found in the Supplementary material . The following reviewed papers are presented in ascending order of their published date (Aellen et al, 2021 ; Asheri et al, 2021 ; Ashwini and Nagaraj, 2021 ; Awais et al, 2021 ; Cai et al, 2021 ; Dagdevir and Tokmakci, 2021 ; De Venuto and Mezzina, 2021 ; Du et al, 2021 ; Fan et al, 2021 , 2022 ; Ferracuti et al, 2021 ; Gao N. et al, 2021 ; Gao Z. et al, 2021 ; Gaur et al, 2021 ; Lashgari et al, 2021 ; Lian et al, 2021 ; Liu and Jin, 2021 ; Liu and Yang, 2021 ; Liu et al, 2021 ; Qi et al, 2021 ; Rashid et al, 2021 ; Sun et al, 2021 ; Varsehi and Firoozabadi, 2021 ; Vega et al, 2021 ; Vorontsova et al, 2021 ; Wahid and Tafreshi, 2021 ; Wang and Quan, 2021 ; Xu C. et al, 2021 ; Xu F. et al, 2021 ; Yin et al, 2021 ; Zhang K. et al, 2021 ; Zhang Y. et al, 2021 ; Algarni et al, 2022 ; Ali et al, 2022 ; Asadzadeh et al, 2022 ; Ayoobi and Sadeghian, 2022 ; Bagchi and Bathula, 2022 ; Chang et al, 2022 ; Chen J. et al, 2022 ; Chen L. et al, 2022 ; Cui et al, 2022 ; Geng et al, 2022 ; Islam et al, 2022 ; Jia et al, 2022 ; Kim et al, 2022 ; Ko et al, 2022 ; Li and Sun, 2022 ; Li H. et al, 2022 ; Lin et al, 2022 ; Li Q. et al, 2022 ; Lu et al, 2022 ; Ma et al, 2022 ; Mattioli et al, 2022 ;...…”

Closed-loop motor imagery EEG simulation for brain-computer interfaces

“…The proposed approach, much like other available ground-up EEG simulations, requires users to design a parameterization method of EEG and a neural activity encoding model that fits their purpose of use. The alternative would be the use of endpoint-focused approaches to EEG simulation, such as generational adversarial networks (GANs) and variational autoencoders (VAEs) (Zhang and Liu, 2018 ; Aznan et al, 2019 ; Bao et al, 2021 ; Fahimi et al, 2021 ; Kunanbayev et al, 2021 ; Ko et al, 2022 ) and waveform decomposition and reconstruction-with-noise techniques (Yeung et al, 2004 ; Lotte, 2011 ; Bridwell et al, 2016 ; Dinarès-Ferran et al, 2018 ). Such approaches pay less attention to ensuring that the generative model is consistent with our understanding of the origin of the EEG and its features, or the neurophysiology of BCI control, in exchange for highly realistic EEG.…”

“…Importantly, closed-loop BCI simulators are distinguished from the relatively abundant offline simulation methods of neural activity data (Lotte, 2011 ; Aine et al, 2012 ; Krol et al, 2018 ; Dinarès-Ferran et al, 2018 ; Lindgren et al, 2018 ; Zhang and Liu, 2018 ; Aznan et al, 2019 ; Barzegaran et al, 2019 ; Fahimi et al, 2021 ; Kunanbayev et al, 2021 ; Ko et al, 2022 ) which serve important but different purposes [mainly for neural activity data augmentation to improve BCI decoder training and/or testing (Marturano et al, 2020 ; Ramírez Torres and Daly, 2021 ), or, more rarely, to supplement the development and evaluation of source modeling methods (Aine et al, 2012 ; Gramfort et al, 2013 )].…”

Subject-Independent Deep Architecture for EEG-Based Motor Imagery Classification

“…Then, the network was re-trained on all labeled and pseudo-labeled samples. Generative models were used to generate synthetic data and learn the distributional characteristics of EEG data [37]. The Π model was proposed to deal with the case where the number of labeled samples was limited enough making the pseudo labeling unstable [42].…”