Left Motor δ Oscillations Reflect Asynchrony Detection in Multisensory Speech Perception

Abstract

During multisensory speech perception, slow d oscillations (;1-3 Hz) in the listener's brain synchronize with the speech signal, likely engaging in speech signal decomposition. Notable fluctuations in the speech amplitude envelope, resounding speaker prosody, temporally align with articulatory and body gestures and both provide complementary sensations that temporally structure speech. Further, d oscillations in the left motor cortex seem to align with speech and musical beats, suggesting their possible role in the temporal structuring of (quasi)-rhythmic stimulation. We extended the role of d oscillations to audiovisual asynchrony detection as a test case of the temporal analysis of multisensory prosody fluctuations in speech. We recorded Electroencephalograph (EEG) responses in an audiovisual asynchrony detection task while participants watched videos of a speaker. We filtered the speech signal to remove verbal content and examined how visual and auditory prosodic features temporally (mis-)align. Results confirm (1) that participants accurately detected audiovisual asynchrony, and (2) increased d power in the left motor cortex in response to audiovisual asynchrony. The difference of d power between asynchronous and synchronous conditions predicted behavioral performance, and (3) decreased d-b coupling in the left motor cortex when listeners could not accurately map visual and auditory prosodies. Finally, both behavioral and neurophysiological evidence was altered when a speaker's face was degraded by a visual mask. Together, these findings suggest that motor d oscillations support asynchrony detection of multisensory prosodic fluctuation in speech.

Speech listening entails neural encoding of invisible articulatory features

“…Importantly, we found a spectral dissociation that emerged in the frequency-resolved PID analysis. Lower frequencies (from ∼0.5 to 4 Hz) appear to be relevant for kinematics-specific information (unique information provided by the PC1), in agreement with previous evidence that entrainment in the delta-range originates from higher-level processes in frontal ( Molinaro et al, 2016 ) and motor cortices in particular ( Park et al, 2015 ;Morillon et al, 2019 ;Biau et al, 2022 ). Instead, higher frequencies (between 4 and 8 Hz) contain unique information carried by the speech envelope.…”

Activating words without language: Beta and theta oscillations reflect lexical access and control processes during verbal and non-verbal object recognition tasks

“…We used cross-frequency coupling analysis to evaluate the functional relationship between theta phase and beta amplitude during picture-object processing in the two tasks. In detail, we determined the participant-specific peaks in theta and beta power from the regions and timewindows of interest identified in the previous analyses, and we used the modulation index (Tort et al 2010;Biau et al 2022) to approximate theta-beta PAC (Figure 3b and 3c).…”

“…In detail, we determined the participant-specific peaks in theta and beta power from the regions and time-windows of interest identified in the previous analyses, and we used the modulation index (Tort et al . 2010; Biau et al . 2022) to approximate theta-beta PAC ( Figure 3b and 3c ).…”

“…To assess the extent to which the beta activity coupled to theta phase, we calculated the modulation index (MI) from the correct trials only (Tort et al 2010; Griffiths et al . 2021; Biau et al 2022). First, the peaks in the theta and beta frequency bands were calculated by estimating power across all electrodes from the regions of interest in the picture-naming task (NamingNC and NamingC) and the size-judgment task (Size-judgmentNC and Size-judgmentC), with the same time-frequency decomposition method as above.…”

T-Rex: sTandalone Recorder of EXperiments; An easy and versatile neural recording platform

“…For a movement experiment ([12], [13], [14], [15], [16]), the streams recorded could be the neural amplifier, experimental triggers, and a movement tracker could be potentially added ([17], [18], [19]) or a force sensor ([20]. For speech perception ([21], [22], [23]) or auditory perception ([24], [25]) the audio stream, experiment triggers and neural data. For speech production ([26], [27], [28], [29]), the streams could be neural data, microphone, and triggers.…”