Cortical encoding of rhythmic kinematic structures in biological motion

Shen, Li; Lu, Xiqian; Yuan, Xiangyong; Hu, Ruichen; Wang, Ying; Jiang, Yi

doi:10.1016/j.neuroimage.2023.119893

Cited by 7 publications

(17 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such linear integration might result from concurrent, independent processing of unisensory inputs without additional interaction of them (Stein et al, 2009). In contrast, at gait-cycle frequency, the congruent audiovisual signals led to a super-additive multisensory enhancement over the linear combination of auditory and visual conditions (AV > A+V), despite that there was no evident cortical tracking effect in the visual condition, different from previous findings obtained with a motion-relevant change detection task (Shen, Lu, Yuan, et al, 2023). This multisensory enhancement may bring about decreased thresholds of detection and identification (Stanford et al, 2005), allowing us to achieve a more clear and stable perception of the external environment and detect weak stimulus changes in time and respond adaptively.…”

Section: Discussioncontrasting

confidence: 99%

“…1a ). During walking, each step of the left or right foot occurs alternatively to form a step cycle, and the antiphase oscillations of limbs during two steps characterize a gait cycle (Shen, Lu, Yuan, et al, 2023). In Experiment 1a, the frequency of a full gait cycle is 1 Hz, and the step-cycle frequency is 2 Hz.…”

Section: Resultsmentioning

confidence: 99%

“…In a broad sense, the current study deepens our understanding of the neural processing of audiovisual signals in natural stimuli with complex temporal structures. Cortical entrainment can track simple rhythmic stimuli like tone sequences or luminance-varying patches (C. Keitel et al, 2017; Yuan et al, 2021) as well as complex rhythmic structures in speech (Brookshire et al, 2017; Ding et al, 2016; Keitel et al, 2018) and BM (Shen, Lu, Yuan, et al, 2023). Beyond unisensory processing, cortical entrainment also plays a role in the multisensory processing of simple or discrete rhythmic signals generated by physical stimulation (Bauer et al, 2021; Miller et al, 2013; Nozaradan et al, 2012b; Simon & Wallace, 2017).…”

Section: Discussionmentioning

confidence: 99%

“…For example, the human locomotion movement has a narrower integration window consisting of each step (i.e., step cycle) and a broader integration window incorporating the opponent motion of the two feet (i.e., gait cycle). A recent study suggests that neural entrainment to these hierarchical kinematic structures contributes to the spatiotemporal integration of visual BM cues in different manners (Shen, Lu, Yuan, et al, 2023). However, it remains open whether and how the cortical tracking of rhythmic signals underpins the AVI of BM information.…”

Section: Introductionmentioning

confidence: 99%

“…Experiment 2 further examined to what extent the AVI effect was specific to the multisensory processing of BM by using non-BM (inverted visual stimuli) as a control. Inversion disrupts the unique, gravity-compatible kinematic features of BM but not the rhythmic signals generated by low-level motion cues (Ma et al, 2022; Shen, Lu, Yuan, et al, 2023; Simion et al, 2008; Troje & Westhoff, 2006; Wang et al, 2022), thus is expected to interfere with the BM-specific neural responses. Participants perceived the visual stimuli accompanied by temporally congruent or incongruent BM sounds.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Hierarchical cortical entrainment orchestrates the multisensory processing of biological motion

Shen,

Li,

Tian

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

When observing others' behaviors, we continuously integrate their movements with the corresponding sounds to achieve efficient perception and develop adaptive responses. However, how human brains integrate these complex audiovisual cues based on their natural temporal correspondence remains unknown. Using electroencephalogram, we demonstrated that cortical oscillations entrained to hierarchical rhythmic structures in audiovisually congruent human walking movements and footstep sounds. Remarkably, the entrainment effects at different time scales exhibit distinct modes of multisensory integration, i.e., an additive integration effect at a basic-level integration window (step-cycle) and a super-additive multisensory enhancement at a higher-order temporal integration window (gait-cycle). Moreover, only the cortical tracking of higher-order rhythmic structures is specialized for the multisensory integration of human motion signals and correlates with individuals' autistic traits, suggesting its functional relevance to biological motion perception and social cognition. These findings unveil the multifaceted roles of entrained cortical activity in the multisensory perception of human motion, shedding light on how hierarchical cortical entrainment orchestrates the processing of complex, rhythmic stimuli in natural contexts.

show abstract

Section: Discussioncontrasting

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Hierarchical cortical entrainment orchestrates the multisensory processing of biological motion

Shen,

Li,

Tian

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Asymmetrical sequence chunking for auditory and visual domains in the left and right hemispheres

Ding,

Lu,

et al. 2023

Preprint

View full text Add to dashboard Cite

Chunking is an effective mechanism to process sensory sequences. It remains unclear, however, whether the chunking mechanism is domain-general or specific to each input sensory modality. Here, we investigate the functions of the Broca’s area and its right homolog mediating chunking when human participants break up auditory and visual sequences into chunks based on rules. Using magnetoencephalography (MEG), we demonstrate that neural activity can track the rhythms of sensory chunks across the brain areas. Critically, the precision of chunk-tracking activity in the left and right inferior frontal gyri (IFG) separately predicts the behavioral performance for auditory and visual chunking. Transcranial magnetic stimulation (TMS) further reveals that deactivating the left and right IFG distinctively influences the behavioral performance of auditory and visual chunking. Additionally, structural imaging shows that the strength of the left and right arcuate fasciculi, connecting frontal and temporal cortices, predicts both ipsilateral IFG activation and modality-dependent chunking performance. The consistent electrophysiological, neural modulation and structural connectivity results collectively suggest distinct hemispherical lateralized temporal-frontal networks for rule-based chunking in the auditory and visual domains.

show abstract

Sensorimotor Account of Multimodal Prosody

Coulson¹

2023

Preprint

View full text Add to dashboard Cite

The temporal signatures that characterize speech—especially its prosodic qualities—are observable in the movements of the hands and bodies of its speakers. A neurobiological account of these prosodic rhythms is thus likely to benefit from insights on the neural coding principles underlying co-speech gestures. Here we consider a role for the information processing capacity of the vestibular system, a sensory system that encodes movements of the body, in the neural representation of prosody. Careful review of the vestibular system’s functional organization as a hierarchical and predictive system, its relevance for rhythmic sound sequences, dynamic attention, and its involvement in vocalization suggests vestibular codes may be important for the neural tracking of speech. As the kinematics and time course of co-speech movements mirror prosodic fluctuations in spoken prosody, we argue that the vestibular system helps encode and decode these prosodic features in multimodal discourse and possibly during auditory speech processing alone.

show abstract

Cortical encoding of rhythmic kinematic structures in biological motion

Cited by 7 publications

References 85 publications

Hierarchical cortical entrainment orchestrates the multisensory processing of biological motion

Hierarchical cortical entrainment orchestrates the multisensory processing of biological motion

Asymmetrical sequence chunking for auditory and visual domains in the left and right hemispheres

Sensorimotor Account of Multimodal Prosody

Contact Info

Product

Resources

About