Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG

Abstract

Classification of electroencephalography (EEG) signals corresponding to imagined speech production is important for the development of a direct-speech brain–computer interface (DS-BCI). Deep learning (DL) has been utilized with great success across several domains. However, it remains an open question whether DL methods provide significant advances over traditional machine learning (ML) approaches for classification of imagined speech. Furthermore, hyperparameter (HP) optimization has been neglected in DL-EEG studies, resulting in the significance of its effects remaining uncertain. In this study, we aim to improve classification of imagined speech EEG by employing DL methods while also statistically evaluating the impact of HP optimization on classifier performance. We trained three distinct convolutional neural networks (CNN) on imagined speech EEG using a nested cross-validation approach to HP optimization. Each of the CNNs evaluated was designed specifically for EEG decoding. An imagined speech EEG dataset consisting of both words and vowels facilitated training on both sets independently. CNN results were compared with three benchmark ML methods: Support Vector Machine, Random Forest and regularized Linear Discriminant Analysis. Intra- and inter-subject methods of HP optimization were tested and the effects of HPs statistically analyzed. Accuracies obtained by the CNNs were significantly greater than the benchmark methods when trained on both datasets (words: 24.97%, p < 1 × 10–7, chance: 16.67%; vowels: 30.00%, p < 1 × 10–7, chance: 20%). The effects of varying HP values, and interactions between HPs and the CNNs were both statistically significant. The results of HP optimization demonstrate how critical it is for training CNNs to decode imagined speech.

Decoding Covert Speech From EEG-A Comprehensive Review

“…They mean up, down, left, right, forward and backward, respectively. Similar to García et al ( 2012 ) and García-Salinas et al ( 2019 ), Cooney et al ( 2020 ) have also used Spanish words. The six Spanish words used by Cooney et al ( 2020 ) are “arriba”, “abajo”, “derecha”, “izquierda”, “adelante”, and “atrás” which mean up, down, left, right, backward, and forward.…”

“…Similar to García et al ( 2012 ) and García-Salinas et al ( 2019 ), Cooney et al ( 2020 ) have also used Spanish words. The six Spanish words used by Cooney et al ( 2020 ) are “arriba”, “abajo”, “derecha”, “izquierda”, “adelante”, and “atrás” which mean up, down, left, right, backward, and forward. García et al ( 2012 ), García-Salinas et al ( 2019 ) made use of the same dataset acquired using -channel Emotiv EPOC commercial grade EEG acquisition system sampled at 128 Hz.…”

“…In a very recent work by Cooney et al ( 2020 ), imagined speech is classified using three different CNN architectures that take the temporal domain EEG signals as the input. The aim of the work was to study the influence of hyperparameter optimization in decoding imagined speech.…”

“…The EEG data for Pawar and Dhage ( 2020 ) is acquired using 64-channel Neuroscan synamps 2 research grade EEG acquisition system sampled at 1,000 Hz. Koizumi et al ( 2018 ) used a 65-channel EEG-1200, Nihon Kohden Corporation research grade EEG acquisition system sampled at 1,000 Hz whereas Cooney et al ( 2020 ) used the dataset acquired using 18-channel Grass 8-18-36 commercial grade EEG acquisition system sampled at 1,024 Hz. The κ values of these systems are given in Table 5 (sl.…”

“…Clearly, Koizumi et al ( 2018 ) has the best performance in terms of κ value and Cooney et al ( 2020 ) has the worst performance. This cannot be attributed to the system type (commercial grade/research grade) because García-Salinas et al ( 2019 ), who also made use of a commercial grade system like Cooney et al ( 2020 ), have obtained much better performance than Cooney et al ( 2020 ). Also, the data sampling rate may not have affected the accuracy.…”