Audiovisual phonological mismatch produces early negativity in auditory cortex

Abstract

During silent reading, visual information provided by letters is converted to auditory information in the mind. The purpose of this study was to identify the primary locus for auditory verbal imagery in the brain. Neuromagnetic recording was obtained from 10 right-handed study participants, who were instructed to identify infrequently occurring phonological mismatches between a random-ordered sequence of syllable sounds and a visually presented syllabogram sequence. The activity difference in early latency, calculated by subtracting the averaged responses to matched syllables from the averaged responses to mismatched syllables, showed a spatiotemporal profile strikingly similar to that of mismatch negativity. Auditory imagery of forthcoming verbal sounds may establish a memory trace as a template for imagery-based mismatch negativity generation in the auditory cortex.

Evidence for Training-Induced Plasticity in Multisensory Brain Structures: An MEG Study

“…The PCG activation (BA 44) that was found in the right hemisphere can be attributed to an automatic attraction of the attention by the deviant stimulus, along with the use of working memory during the discrimination process [25]. The audio-visual MMN was generated by the right STG confirming previous results obtained with equivalent current dipoles approach [15], [26]. The fact that the left temporal cortex did not exceed the threshold of significance can be attributed to the typical right lateralization of the auditory MMN with respect to musical stimuli [27].…”

Multisensory Integration during Short-term Music Reading Training Enhances Both Uni- and Multisensory Cortical Processing

“…The right STG activation was not present in our previous study (Paraskevopoulos et al, 2012b). Nevertheless, STG is a region that is well known for responding to audiovisual stimuli (Werner & Noppeney, 2010;Ghazanfar & Schroeder, 2006), and it has been correlated with the identification of audiovisual mismatches (Paraskevopoulos et al, 2012a;Yumoto et al, 2005). It must be noted here that our paradigm requires that the auditory and visual stimulus patterns are jointly processed, but it does not conclusively demonstrate multisensory integration on a neural level.…”

Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex

“…Several studies used standard stimuli in which the auditory and visual dimensions shared a semantic or a symbolic link, for example a hammer hitting a nail (Ullsperger et al, 2006), piano keys producing the corresponding sound (Aoyama et al, 2006), scorelike visual symbol coding the pitch of a tone (Widmann et al, 2004) or graphemes associated with speech sounds (Yumoto et al, 2005). Deviant stimuli were audiovisual entities which departed from the standard only in the visual dimension and provided conflictual information about the (generally upcoming) auditory stimulus.…”