Exploiting Multiple EEG Data Domains with Adversarial Learning

Abstract

Electroencephalography (EEG) is shown to be a valuable data source for evaluating subjects' mental states. However, the interpretation of multi-modal EEG signals is challenging, as they suffer from poor signal-to-noise-ratio, are highly subject-dependent, and are bound to the equipment and experimental setup used, (i.e. domain). This leads to machine learning models often suffer from poor generalization ability, where they perform significantly worse on real-world data than on the exploited training data. Recent research heavily focuses on cross-subject and cross-session transfer learning frameworks to reduce domain calibration efforts for EEG signals. We argue that multi-source learning via learning domain-invariant representations from multiple data-sources is a viable alternative, as the available data from different EEG data-source domains (e.g., subjects, sessions, experimental setups) grow massively. We propose an adversarial inference approach to learn data-source invariant representations in this context, enabling multi-source learning for EEG-based braincomputer interfaces. We unify EEG recordings from different source domains (i.e., emotion recognition datasets SEED, SEED-IV, DEAP, DREAMER), and demonstrate the feasibility of our invariant representation learning approach in suppressing datasource-relevant information leakage by 35% while still achieving stable EEG-based emotion classification performance.

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

“…Data from each modality are applied individually or combined with others. In 374 articles, im- age modality was present in 247, which were image data individually used in 129 articles [49], [51], [52], [54], [56], [57], [59]- [61], [63], [64] [103], [105], [106], [108], [110], [112]- [115], [117], [120]- [123], [128], [130], [134], [135], [137]- [149], [151], [152], [154], [155], [157], [158], [160], [162], [172]- [174], [176], [178], [191], [194]- [196], [198], [202], [203], [209], [210], [212], [214],…”

“…Grammar-based, encoderdecoder, and continuous are three subtypes of generative. Of 19 articles, one was grammar-based [372], twelve were encoder-decoder based [110], [155], [194], [222], [291], [339], [341], [342], [371], [373], [374], [417], and six were continuous-based [129], [177], [243], [294], [344], [346].…”

Subject-invariant feature learning for mTBI identification using LSTM-based variational autoencoder with adversarial regularization

“…The positive impact of adversarial networks in learning generalizable representations that are domain-, task-, subjectsand source-invariant has been shown recently in many applications such as in drug molecular analysis (Hong et al, 2019), decoding brain states (Du et al, 2019), brain lesion segmentation (Kamnitsas et al, 2017), and evaluating subjects' mental states (Bethge et al, 2022b). These methods learn representations that are independent of some nuisance variables such as subject-specific or task-specific variations.…”