Learning to operate an imagined speech Brain-Computer Interface involves the spatial and frequency tuning of neural activity

Abstract

Brain-Computer Interfaces (BCI) will revolutionize the way people with impaired speech production can communicate. While recent studies confirm the possibility of decoding imagined speech based on pre-recorded intracranial neurophysiological signals, current efforts focus on collecting vast amounts of data to train classifiers, rather than exploring how the individual’s brain adapts to improve BCI control, an important aspect given the known problem of “BCI illiteracy”, the inability of some individuals to operate a BCI. This issue can be investigated by providing real-time feedback to allow users to identify the best control strategy. In this study, we trained 15 healthy participants to operate a simple binary BCI system based on electroencephalography (EEG) signals through syllable imagery for five consecutive days. We explored whether BCI-control improves with training and characterized the underlying neural dynamics, both in terms of EEG power changes and of the neural features contributing to real-time classification. Despite considerable interindividual variability in performance and learning, a significant improvement in BCI control was observed from day 1 to 5. Performance improvement was reproducibly associated with a global EEG power increase in frontal theta and a focal increase in temporal low-gamma, showing that learning to operate an imagined-speech BCI involves global and local dynamical changes involving low- and high-frequency neural features, respectively. These findings indicate that both machine and human learning must be considered to reach optimal controllability of imagined-speech BCI, and that non-invasive BCI-learning can help predict the individual benefit from an invasive speech BCI and guide both the electrode implantation and decoding strategies.

Adaptive LDA classifier enhances real-time control of an EEG Brain-computer interface for imagined-speech decoding

“…First, we identified the optimal UC ∑ and UC μ values (i.e. a UC-pair) to be used during real-time BCI-control by performing simulation analysis on a previously recorded dataset employing the same experimental paradigm as the one tested in the present study (33) We based the choice of the UC-pair on three distinct metrics: “ Modulated PA ”, “ Consistency Score ”, and “ Modulated Bias ”. The “ Modulated PA ” matrix, obtained as the difference in PA between static and adaptive classification, indicated that combinations yielding the strongest increase in PA (amounting to 5%) with the adaptive classifier fall within a restricted UC range (UC μ : 0.4*2 -3 to 0.4*2 -4 , UC ∑ : 0.4*2 -3 to 0.4*2 -1 , Figure 3a).…”

“…We first used EEG data from a previous study (Study 1) conducted in our laboratory (33) , comprising 15 healthy volunteers (5 women; average age: 23.9 years, SD ± 2.3, age range: 19-29 years) to conduct decoding simulations described in Section 2.5 “Simulation analysis”. In a second step (Study 2), we recruited twenty healthy participants (13 women; average age: 25.2 years, SD ±3.1, age range: 20-30 years).…”

“…The BCI loop was developed upon the NeuroDecode framework (Fondation Campus Biotech Geneva, https://github.com/fcbg-hnp/NeuroDecode.git) already employed in previous BCI experiments (33,56,57) .…”

“…x, ∑ -1 , μ x ), UC ∑ and UC μ are constants that need to be pre-specified. To identify the optimal UC values, specific to the syllable imagery task, to be used in the adaptive LDA classifier, we conducted classification simulations on the Study 1 EEG dataset (33) . Importantly, the binary /fɔ/ vs. /gi/ syllable imagery task was exactly the same in Study 1 (optimization) and in Study 2 (static vs. adaptive strategy comparison).…”

“…We checked that there was no difference in the CV accuracy between the two sessions during the offline training as the experimental conditions were the same. We then tested differences in CV accuracy between the offline training and online control , as a previous study using the same paradigm highlighted a higher decoding accuracy when visual feedback was provided (33) . Last, we investigated whether the magnitude of this difference depended on the classifier used (static vs. adaptive).…”