The Role of Audiovisual Speech in the Early Stages of Lexical Processing as Revealed by the ERP Word Repetition Effect

Basirat, Anahita; Brunellière, Angèle; Hartsuiker, Robert J.

doi:10.1111/lang.12265

Cited by 13 publications

(17 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, Basirat et al (2018) found that this repetition effect on the P200 interacted with multisensory context. For the initial word presentation, audio-visual words were associated with a smaller ERP than were the audio-only words, suggesting that the visual context facilitated lexical access (see Basirat et al, 2018 for discussion). Their results indicate that the multisensory information of audio-visual speech may facilitate word processing analogously to the facilitation provided by word repetition.…”

Section: Discussionmentioning

confidence: 85%

“…The P200 ERP component is known to be modulated by word repetition (e.g., Almeida & Poeppel, 2013). However, Basirat et al (2018) found that this repetition effect on the P200 interacted with multisensory context. For the initial word presentation, audio-visual words were associated with a smaller ERP than were the audio-only words, suggesting that the visual context facilitated lexical access (see Basirat et al, 2018 for discussion).…”

Section: Neurophysiological Processing Of Multisensory Speechmentioning

confidence: 90%

“…However, Basirat et al (2018) have used the word repetition effect in a recent EEG study to examine the effects of multisensory and linguistic processes. The word repetition effect is the finding that prior processing of words, but not nonwords, facilitates subsequent processing of those same words (e.g., participants will identify a word faster the second time it is presented; e.g., Forbach et al, 1974).…”

Section: Neurophysiological Processing Of Multisensory Speechmentioning

confidence: 99%

See 2 more Smart Citations

Selective adaptation in speech: Measuring the effects of visual and lexical contexts.

Dorsi

Rosenblum

Samuel

et al. 2021

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

Speech selective adaptation is a phenomenon in which repeated presentation of a speech stimulus alters subsequent phonetic categorization. Prior work has reported that lexical, but not multisensory, context influences selective adaptation. This dissociation suggests that lexical and multisensory contexts influence speech perception through separate and independent processes (see Samuel & Lieblich, 2014). However, this dissociation is based on results reported by different studies using different stimuli. This leaves open the possibility that the divergent effects of multisensory and lexical contexts on selective adaptation may be the result of idiosyncratic differences in the stimuli rather than separate perceptual processes. The present investigation used a single stimulus set to compare the selective adaptation produced by lexical and multisensory contexts. In contrast to the apparent dissociation in the literature, we find that multisensory information can in fact support selective adaptation. Public Significance StatementThis work challenges prior findings that indicate that low level speech processes precede the perceptual integration of auditory and visual information. In doing so, this research suggests that auditory and visual information are combined early in perception. This research could be beneficial for perceptual training to improve verbal communication.

show abstract

Section: Discussionmentioning

confidence: 85%

Section: Neurophysiological Processing Of Multisensory Speechmentioning

confidence: 90%

Section: Neurophysiological Processing Of Multisensory Speechmentioning

confidence: 99%

See 1 more Smart Citation

Selective adaptation in speech: Measuring the effects of visual and lexical contexts.

Dorsi

Rosenblum

Samuel

et al. 2021

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

show abstract

“…An increasing body of studies has explored the mechanisms underlying audiovisual integration of letters and speech sounds [7][8][9] and consistently demonstrated the superiority of audio-visual integration over processing letters and speech sounds separate [10,11]. For example, a magnetoencephalography (MEG) study showed enhanced brain activity predominantly in the right temporal-occipital-parietal junction and the left and right superior temporal sulci for audio-visual integration of phonemes and graphemes [12].…”

Section: Introductionmentioning

confidence: 99%

The different brain areas occupied for integrating information of hierarchical linguistic units: a study based on EEG and TMS

Pei

Yuan

et al. 2022

Cerebral Cortex

View full text Add to dashboard Cite

Human language units are hierarchical, and reading acquisition involves integrating multisensory information (typically from auditory and visual modalities) to access meaning. However, it is unclear how the brain processes and integrates language information at different linguistic units (words, phrases, and sentences) provided simultaneously in auditory and visual modalities. To address the issue, we presented participants with sequences of short Chinese sentences through auditory, visual, or combined audio-visual modalities while electroencephalographic responses were recorded. With a frequency tagging approach, we analyzed the neural representations of basic linguistic units (i.e. characters/monosyllabic words) and higher-level linguistic structures (i.e. phrases and sentences) across the 3 modalities separately. We found that audio-visual integration occurs in all linguistic units, and the brain areas involved in the integration varied across different linguistic levels. In particular, the integration of sentences activated the local left prefrontal area. Therefore, we used continuous theta-burst stimulation to verify that the left prefrontal cortex plays a vital role in the audio-visual integration of sentence information. Our findings suggest the advantage of bimodal language comprehension at hierarchical stages in language-related information processing and provide evidence for the causal role of the left prefrontal regions in processing information of audio-visual sentences.

show abstract

Section: Introductionmentioning

confidence: 99%

The different brain areas occupied for integrating information of hierarchical linguistic units: a study based on EEG and TMS

Pei

Yuan

et al. 2021

Preprint

View full text Add to dashboard Cite

Human linguistic units are hierarchical, and our brain responds differently when processing linguistic units during sentence comprehension, especially when the modality of the received signal is different (auditory, visual, or audio-visual). However, it is unclear how the brain processes and integrates language information at different linguistic units (words, phrases, and sentences) provided simultaneously in audio and visual modalities. To address the issue, we presented participants with sequences of short Chinese sentences through auditory or visual or combined audio- visual modalities, while electroencephalographic responses were recorded. With a frequency tagging approach, we analyzed the neural representations of basic linguistic units (i.e., characters/monosyllabic words) and higher-level linguistic structures (i.e., phrases and sentences) across the three modalities separately. We found that audio-visual integration occurs at all linguistic units, and the brain areas involved in the integration varied across different linguistic levels. In particular, the integration of sentences activated the local left prefrontal area. Therefore, we used continuous theta-burst stimulation (cTBS) to verify that the left prefrontal cortex plays a vital role in the audio-visual integration of sentence information. Our findings suggest the advantage of bimodal language comprehension at hierarchical stages in language-related information processing and provide evidence for the causal role of the left prefrontal regions in processing information of audio-visual sentences.

show abstract

The Role of Audiovisual Speech in the Early Stages of Lexical Processing as Revealed by the ERP Word Repetition Effect

Cited by 13 publications

References 44 publications

Selective adaptation in speech: Measuring the effects of visual and lexical contexts.

Selective adaptation in speech: Measuring the effects of visual and lexical contexts.

The different brain areas occupied for integrating information of hierarchical linguistic units: a study based on EEG and TMS

The different brain areas occupied for integrating information of hierarchical linguistic units: a study based on EEG and TMS

Contact Info

Product

Resources

About