Brain–Computer Interface With Language Model–Electroencephalography Fusion for Locked-In Syndrome

Abstract

Background Some non-invasive brain computer interface (BCI) systems are currently available for locked-in syndrome (LIS) but none have incorporated a statistical language model during text generation. Objective To begin to address the communication needs of individuals with LIS using a non-invasive BCI that involves Rapid Serial Visual Presentation (RSVP) of symbols and a unique classifier with EEG and language model fusion. Methods The RSVP Keyboard™ was developed with several unique features. Individual letters are presented at 2.5 per sec. Computer classification of letters as targets or non-targets based on EEG is performed using machine learning that incorporates a language model for letter prediction via Bayesian fusion enabling targets to be presented only 1–4 times. Nine participants with LIS and nine healthy controls were enrolled. After screening, subjects first calibrated the system, and then completed a series of balanced word generation mastery tasks that were designed with five incremental levels of difficulty, that increased by selecting phrases for which the utility of the language model decreased naturally. Results Six participants with LIS and nine controls completed the experiment. All LIS participants successfully mastered spelling at level one and one subject achieved level five. Six of nine control participants achieved level five. Conclusions Individuals who have incomplete LIS may benefit from an EEG-based BCI system, which relies on EEG classification and a statistical language model. Steps to further improve the system are discussed.

Teaching mindfulness meditation to adults with severe speech and physical impairments: An exploratory study

“…A convenience sample of participants was recruited from an ongoing BCI communication study for people with SSPI (Fried-Oken, Mooney, Peters, & Oken, 2013; B. S. Oken, et al, 2013; Orhan, et al, 2013).…”

Language-Model Assisted Brain Computer Interface for Typing: A Comparison of Matrix and Rapid Serial Visual Presentation

“…After calibration, each session proceeded with the mastery task [28] followed by the copy phrase task with 8 sentences. We use level 1 mastery task to familiarize the users with the copy phrase task.…”

Innovations for Supporting Communication: Opportunities and Challenges for People with Complex Communication Needs

“…One of the longest-established noninvasive BCIs for communication is the visual P300 grid speller [45], which uses event-related potential to track responses to novel stimuli to select items on a communication interface – in this instance letters from within a grid. Variations on this interface (e.g., the Rapid Serial Visual Presentation Speller ) have been used to support spelling by adults with locked-in syndrome [43], indicating that it is becoming possible to provide access to communication even for individuals with profound impairments. However, the application of these technologies in AAC is still at a very early stage.…”