Reading-induced shifts of perceptual speech representations in auditory cortex

Abstract

Learning to read requires the formation of efficient neural associations between written and spoken language. Whether these associations influence the auditory cortical representation of speech remains unknown. Here we address this question by combining multivariate functional MRI analysis and a newly-developed ‘text-based recalibration’ paradigm. In this paradigm, the pairing of visual text and ambiguous speech sounds shifts (i.e. recalibrates) the perceptual interpretation of the ambiguous sounds in subsequent auditory-only trials. We show that it is possible to retrieve the text-induced perceptual interpretation from fMRI activity patterns in the posterior superior temporal cortex. Furthermore, this auditory cortical region showed significant functional connectivity with the inferior parietal lobe (IPL) during the pairing of text with ambiguous speech. Our findings indicate that reading-related audiovisual mappings can adjust the auditory cortical representation of speech in typically reading adults. Additionally, they suggest the involvement of the IPL in audiovisual and/or higher-order perceptual processes leading to this adjustment. When applied in typical and dyslexic readers of different ages, our text-based recalibration paradigm may reveal relevant aspects of perceptual learning and plasticity during successful and failing reading development.

Brain activity patterns of phonemic representations are atypical in beginning readers with family risk for dyslexia

“…This is in line with other studies showing that auditory cortical processing of phonemes follows a protracted development up to adolescence (Bonte, Ley, Scharke, & Formisano, ; Pang & Taylor, ; Sharma, Kraus, Mcgee, & Nicol, ; Sowell et al, ). In addition, it has been demonstrated that learning to read induces changes in the speech perception network (Dehaene et al, ), and, of specific importance, in the neural representations of phonemes, at least temporarily, as evidenced by reading‐induced shifts in the neural representations of ambiguous speech sounds (Bonte et al, ). Therefore, investigating the role of phonemic representations in dyslexia should happen early in reading development, when these representations have not yet been impacted by the amount and the efficiency of reading.…”

“…Given the focused activation maps when listening to the speech sounds (see Figure for whole brain univariate analyses) and given that our FOV in functional scans did not cover the full brain in all children (see section Data Acquisition), fMRI analyses were restricted to superior temporal regions (ST mask). To this end, we created a left and right ST mask (see left panel of Figure ) which was previously observed to encompass univariate responses to short meaningless speech sounds (Bonte et al, ), as well as to contain vowel (Bonte et al, ; Formisano et al, ) and syllable (Bonte, Correia, Keetels, Vroomen, & Formisano, ; Kilian‐Hutten, Valente, Vroomen, & Formisano, ) representations.…”

Multisensory integration of speech sounds with letters vs. visual speech: only visual speech induces the mismatch negativity

“…() and Bonte et al . (), an ambiguous speech sound halfway between /aba/ and /ada/ was coupled repeatedly with text (‘aba’ or ‘ada’). In an auditory‐only post‐test following this adaptation phase, it appeared that text had shifted the interpretation of the ambiguous sound towards /aba/ or /ada/, respectively.…”

Recalibration of vocal affect by a dynamic face

“…It is known that phonetic recalibration is sub-served by a brain network that includes the superior temporal sulcus (STS, see Bonte et al 2017; Kilian-Hutten et al 2011), which is also involved in audiovisual integration of emotional affect (e.g., Ethofer et al 2006; Hagan et al 2009; Klasen et al 2011). Given that comprehension of vocal affect is driven by bilateral mechanisms that involve a myriad of sensory, cognitive, and emotional processing systems (Schirmer and Kotz 2006), it may well be that STS also has a functional role in recalibration of emotional valence.…”