Real-time control of a hearing instrument with EEG-based attention decoding

Abstract

Restoring normal speech perception in everyday noisy acoustic environments remains an outstanding challenge for hearing aids. Speech separation technology is improving rapidly but hearing instrument technology cannot fully exploit this advance without knowing which sound sources the user wants to hear. Even with high-quality source separation, the hearing aid must know which speech streams to enhance and which to suppress. Advances in EEG-based decoding of auditory attention raise the potential of a neuro-steered hearing instrument that selectively enhances the sound sources that a hearing-impaired listener is focusing their attention on. Here, we present a real-time brain-computer interface (BCI) system implementing this concept. Our system combines a stimulus-response model based on canonical correlation analysis (CCA) for real-time EEG attention decoding with a multi-microphone hardware platform enabling low-latency real-time speech separation through spatial beamforming. In this paper, we provide an overview of the system and its various components and discuss prospects and limitations of the technology. We illustrate its application with case studies of listeners steering acoustic feedback of competing speech streams via real-time attention decoding. A software implementation code of the system is publicly available for further research and explorations.