Auditory detection is modulated by theta phase of silent lip movements

Biau, Emmanuel; Wang, Danying; Park, Hyojin; Jensen, Ole; Hanslmayr, Simon

doi:10.1016/j.crneur.2021.100014

Cited by 15 publications

(17 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been proposed that visual cues reset auditory delta/theta dynamics to prepare them for upcoming acoustic information (Biau et al, 2021;Mégevand et al, 2020;Thorne & Debener, 2014). Processing of audio-visual speech information leads to shorter neural responses latencies than those evoked by auditory-only speech (van Wassenhove et al, 2005), and the presentation of distinct acoustic and visual consonantal information can lead to the percept of a third consonant (Mcgurk & Macdonald, 1976).…”

Section: E Cross-modality and Sensory-motor Interactionsmentioning

confidence: 99%

“…For example, visual speech cues reset auditory dynamics in general, not only speech-specific ones (Biau et al, 2021). Some evidence for speech-specific effects has been reported for auditory-motor interactions.…”

Section: B4 Cross-modality and Sensory-motor Interactionsmentioning

confidence: 99%

“…This does not necessarily rule out an involvement of oscillations as they possess means to change their instantaneous frequency and phase. Acoustic "edges" might serve as a cue to "reset" oscillations (Doelling et al, 2014) and visual cues might prepare oscillatory activity for upcoming acoustic information (Biau et al, 2021;Mégevand et al, 2020;Thorne & Debener, 2014). How exactly this is done remains to be investigated, as well as the question whether and why a rhythmic neural process (i.e.…”

Section: B Neural Oscillations Underlying Speech Processing: Open Que...mentioning

confidence: 99%

See 2 more Smart Citations

Neural Dynamics in Speech Processing: Properties, Specificity and Open Questions

Zoefel¹,

Kösem²

2022

Preprint

View full text Add to dashboard Cite

Human speech is undoubtedly a particularly relevant acoustic stimulus for our species, due to its role of information transmission during communication. Previous research has aimed to map speech processing onto macro-anatomical brain structures specifically dedicated to this purpose, an endeavour that turned out to be challenging. A more recent line of research addressed the fact that speech is a dynamic signal, and instead focused on similarly dynamic neural activity. We here focus on the question of whether and how speech is special for such neural dynamics. We review findings that suggest that this is indeed the case: First, properties of human speech are strikingly similar to those found in neural dynamics, indicating that the two co-evolved. Second, neural dynamics produced by human speech have distinct, characteristic properties that are not only an amplified version of those observed during other, non-speech sounds. Although open questions remain, speech seems special for neural dynamics in humans.

show abstract

Section: E Cross-modality and Sensory-motor Interactionsmentioning

confidence: 99%

Section: B4 Cross-modality and Sensory-motor Interactionsmentioning

confidence: 99%

Section: B Neural Oscillations Underlying Speech Processing: Open Que...mentioning

confidence: 99%

See 1 more Smart Citation

Neural Dynamics in Speech Processing: Properties, Specificity and Open Questions

Zoefel¹,

Kösem²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, other sources of interference with putative MMN measurements may emerge as unintended consequences of subject behaviour during passive auditory oddball experiments. For instance, two common approaches are to instruct the subject to watch a silent film or read a book, which present different sets of cues that could unintentionally modulate the auditory response in different respects (Biau et al, 2021;Rayner and Clifton, 2009;Zoefel, 2021). As such, demonstrating that MMN reflects a specific prediction-error signal is challenging, and the balance of evidence from the current study leans decidedly towards the hypothesis that intensity MMN reflects modulation of obligatory sensory components of the ERP.…”

Section: Resultsmentioning

confidence: 82%

Recurrent neural network model of human event-related potentials in response to intensity oddball stimulation

O’Reilly

2022

Preprint

View full text Add to dashboard Cite

The mismatch negativity (MMN) component of the human event-related potential (ERP) is frequently interpreted as a sensory prediction-error signal. However, there is ambiguity concerning the neurophysiology underlying hypothetical prediction and prediction-error signalling components, and whether these can be dissociated from overlapping obligatory components of the ERP that are sensitive to physical properties of sounds. In the present study, a hierarchical recurrent neural network (RNN) was fitted to ERP data from 38 subjects. After training the model to reproduce ERP waveforms evoked by 80 dB standard and 70 dB deviant stimuli, it was used to simulate a response to 90 dB deviant stimuli. Internal states of the RNN effectively combine to generate synthetic ERPs, where individual hidden units are loosely analogous to population-level sources. Model behaviour was characterised using principal component analysis of stimulus condition, layer, and individual unit responses. Hidden units were categorised according to their temporal response fields, and statistically significant differences among stimulus conditions were observed for amplitudes of units peaking in the 0 to 75 ms (P50), 75 to 125 ms (N1), and 250 to 400 ms (N3) latency ranges, surprisingly not including the measurement window of MMN. The model demonstrated opposite polarity changes in MMN amplitude produced by falling (70 dB) and rising (90 dB) intensity deviant stimuli, consistent with loudness dependence of sensory ERP components. Although perhaps less parsimoniously, these observations could be interpreted within the context of predictive coding theory, as examples of negative and positive prediction errors, respectively.

show abstract

“…Importantly, a partial coherence between the left motor region oscillations and lip movements rate have also been identified that directly predicted the participants performance on comprehension, suggesting that motor cortex could facilitate the integration of audiovisual speech through predictive coding and active sensing ( Park et al, 2016 , 2018 ). Several recent studies have proposed that visual cortex entrainment to rhythmic lip motion modulates the responses of auditory cortex via theta phase synchronization ( Crosse et al, 2015 ; Zoefel, 2021 ; see Figure 3 ), including when visual speech only is presented ( Bourguignon et al, 2020 ; Biau et al, 2021 ).…”

Section: Evolution and Development Of Multimodal Integration In The P...mentioning

confidence: 99%

Faces and Voices Processing in Human and Primate Brains: Rhythmic and Multimodal Mechanisms Underlying the Evolution and Development of Speech

2022

View full text Add to dashboard Cite

While influential works since the 1970s have widely assumed that imitation is an innate skill in both human and non-human primate neonates, recent empirical studies and meta-analyses have challenged this view, indicating other forms of reward-based learning as relevant factors in the development of social behavior. The visual input translation into matching motor output that underlies imitation abilities instead seems to develop along with social interactions and sensorimotor experience during infancy and childhood. Recently, a new visual stream has been identified in both human and non-human primate brains, updating the dual visual stream model. This third pathway is thought to be specialized for dynamics aspects of social perceptions such as eye-gaze, facial expression and crucially for audio-visual integration of speech. Here, we review empirical studies addressing an understudied but crucial aspect of speech and communication, namely the processing of visual orofacial cues (i.e., the perception of a speaker’s lips and tongue movements) and its integration with vocal auditory cues. Along this review, we offer new insights from our understanding of speech as the product of evolution and development of a rhythmic and multimodal organization of sensorimotor brain networks, supporting volitional motor control of the upper vocal tract and audio-visual voices-faces integration.

show abstract

Auditory detection is modulated by theta phase of silent lip movements

Cited by 15 publications

References 68 publications

Neural Dynamics in Speech Processing: Properties, Specificity and Open Questions

Neural Dynamics in Speech Processing: Properties, Specificity and Open Questions

Recurrent neural network model of human event-related potentials in response to intensity oddball stimulation

Faces and Voices Processing in Human and Primate Brains: Rhythmic and Multimodal Mechanisms Underlying the Evolution and Development of Speech

Contact Info

Product

Resources

About