Multisensory Integration in Cochlear Implant Recipients

Abstract

Speech perception is inherently a multisensory process involving integration of auditory and visual cues. Multisensory integration in cochlear implant (CI) recipients is a unique circumstance in that the integration occurs following auditory deprivation and the provision of hearing via the CI. Despite the clear importance of multisensory cues for perception in general, and for speech intelligibility specifically, the topic of multisensory perceptual benefits in CI users has only recently begun to emerge as an area of inquiry. We review the research that has been conducted on multisensory integration in CI users to date, and suggest a number of areas needing further research. The overall pattern of results indicates that many CI recipients show at least some perceptual gain that can be attributable to multisensory integration. The extent of this gain, however, varies based on a number of factors, including age of implantation and specific task being assessed (e.g. stimulus detection, phoneme perception, word recognition). Whereas both children and adults with CIs obtain audiovisual benefits for phoneme, word, and sentence stimuli, neither group shows demonstrable gain for suprasegmental feature perception. Additionally, only early-implanted children and the highest performing adults obtain audiovisual integration benefits similar to individuals with normal hearing. Increasing age of implantation in children is associated with poorer gains resultant from audiovisual integration, suggesting a sensitive period in development for the brain networks that subserve these integrative functions, as well as length of auditory experience. This finding highlights the need for early detection of and intervention for hearing loss, not only in terms of auditory perception, but also in terms of the behavioral and perceptual benefits of audiovisual processing. Importantly, patterns of auditory, visual, and audiovisual responses suggest that underlying integrative processes may be fundamentally different between CI users and typical-hearing listeners. Future research, particularly in low-level processing tasks such as signal detection, will help to further assess mechanisms of multisensory integration for individuals with hearing loss, both with and without CIs.

Multisensory Integration and the Society for Neuroscience: Then and Now

“…Trends observed in animals deprived of visual-auditory experience have also been seen in human patients who had similar restrictions due to congenital cataracts or deafness. Following removal of the congenital cataracts, or the introduction of cochlear implants (especially when done early), these individuals developed the ability to integrate visual-auditory cues; albeit the time frame in which they first showed such capabilities is not entirely clear, and they may always have deficits, especially with complex events (e.g., speech) (Stevenson et al, 2017). Whether more rapid and more pronounced enhancements in performance would be induced with a multisensory training paradigm remains to be determined.…”

Experience Creates the Multisensory Transform in the Superior Colliculus

“…They fail to craft the ability to properly synthesize its inputs with those from other modalities (see review by Stein et al, 2014). These defects persist even when later experience is available in a normal housing environment (Xu et al, 2017) and resemble, in a general sense, the multisensory processing abnormalities observed in a number of human psychiatric populations (Brett-Green et al, 2010;Williams et al, 2010;Brandwein et al, 2013;Stevenson et al, 2014Stevenson et al, , 2017Beker et al, 2018). There is significant interest in understanding the mechanisms operating in these defective states.…”

“…Animal studies have suggested that the development of multisensory integration capabilities is shaped by multisensory experience, typically during early life, and that disrupting the acquisition of this experience, or the circuitry needed to properly process that experience, produces defective endpoints (see review by Stein et al, 2014). Anomalous development may help explain the compromised multisensory processing in a number of human populations, contributing to the sensory deficits in Autism Spectrum Disorder, Sensory Processing Disorder, Schizophrenia, and Dyslexia (Brett-Green et al, 2010;Williams et al, 2010;Brandwein et al, 2013;Stevenson et al, 2014Stevenson et al, , 2017Beker et al, 2018).…”

Visual Reliance During Speech Recognition in Cochlear Implant Users and Candidates

“…Several previous studies of AV speech recognition in clinical populations of patients with HL have focused on experienced users of cochlear implants (CIs), devices which restore auditory sensation to the listener (Rabinowitz et al, 1992;Kaiser et al, 2003;Hay McCutcheon et al, 2005;Rouger et al, 2007;Desai et al, 2008;Strelnikov et al, 2009;Altieri et al, 2011;Stevenson et al, 2017;Schreitmüller et al, 2018). For example, several investigators have found that experienced CI users (ECIs) demonstrated stronger reliance on visual cues than NH peers (Desai et al, 2008;Rouger et al, 2008;Leybaert and LaSasso, 2010).…”