The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech

He, Yifei; Gebhardt, Helge; Steines, Miriam; Sammer, Gebhard; Kircher, Tilo; Nagels, Arne; Straube, Benjamin

doi:10.1016/j.neuropsychologia.2015.04.018

Cited by 39 publications

(32 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examining the directional relationship between brain and behavior operationalizes the concept of a brain state, emphasizing the study of largescale oscillatory activity from EEG data to investigate cross-region communication and wholebrain dynamics. Thus, it can test the hypothesis that synchronized frequency oscillations provide a mechanism to form transient networks that can integrate information across local, specialized brain regions (He et al, 2015;Klimesch et al, 2007), and it can reveal how specific oscillations result in behavioral consequences and dynamic fluctuations in task performance. Here, we identified relationships between the temporal dynamics of EEG oscillatory activity and continuous fluctuations in two behavioral measures of driving performance in a simulated environment, the participant's steering behavior and vehicle's heading error.…”

Section: Advancements In Linking Behavior With Brain Activitymentioning

confidence: 99%

“…Often termed state dependency, some researchers have investigated how resting state brain activity modifies incoming information (Wörgötter et al, 1998) or disrupts behavioral performance (Silvanto et al, 2008). Synchronized frequency oscillations are posited as a mechanism to form transient networks that can integrate information across local, specialized brain regions (He et al, 2015;Klimesch et al, 2007), and global brain dynamics characterized by specific frequency oscillations appear to have functional consequences (Buzsáki and Draguhn, 2004). Researchers have identified brain network responses sensitive to task demands (DeSalvo et al, 2014), stimulus properties (Stansbury et al, 2013), and biomarker development for disease (Bassett et al, 2008).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving

et al. 2017

View full text Add to dashboard Cite

Conventional neuroimaging analyses have ascribed function to particular brain regions, exploiting the power of the subtraction technique in fMRI and event-related potential analyses in EEG. Moving beyond this convention, many researchers have begun exploring network-based neurodynamics and coordination between brain regions as a function of behavioral parameters or environmental statistics; however, most approaches average evoked activity across the experimental session to study task-dependent networks. Here, we examined on-going oscillatory activity as measured with EEG and use a methodology to estimate directionality in brain-behavior interactions. After source reconstruction, activity within specific frequency bands (delta: 2-3Hz; theta: 4-7Hz; alpha: 8-12Hz; beta: 13-25Hz) in a priori regions of interest was linked to continuous behavioral measurements, and we used a predictive filtering scheme to estimate the asymmetry between brain-to-behavior and behavior-to-brain prediction using a variant of Granger causality. We applied this approach to a simulated driving task and examined directed relationships between brain activity and continuous driving performance (steering behavior or vehicle heading error). Our results indicated that two neuro-behavioral states may be explored with this methodology: a Proactive brain state that actively plans the response to the sensory information and is characterized by delta-beta activity, and a Reactive brain state that processes incoming information and reacts to environmental statistics primarily within the alpha band.

show abstract

Section: Advancements In Linking Behavior With Brain Activitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Literature suggests that human brain is able to effortlessly integrate semantic information from these two input channels. With functional magnetic resonance imaging (fMRI), it is suggested that two regions in the left hemisphere, the left posterior superior temporal sulcus and the left inferior frontal gyrus are crucially involved in the semantic combination and semantic integration between gesture and verbal speech (Dick et al, 2012;Green et al, 2009;He et al, 2015;He et al, 2018b;Holle et al, 4 actions (reaching, grasping, and etc.) observed from egocentric vs. allocentric view-point elicited greater sensorimotor-related alpha or beta band power decrease (Angelini et al, 2018;Drew et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Gesture’s body orientation modulates the N400 during semantic integration of gesture and visual sentence

Luell

Muralikrishnan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Body orientation of gesture entails social-communicative intention, and may thus influence how gestures are perceived and comprehended together with auditory speech during face-to-face communication. To date, despite the emergence of neuroscientific literature on the role of body orientation on hand action perception, limited studies have directly investigated the role of body orientation in the interaction between gesture and language. To address this research question, we carried out an EEG experiment presenting to participants (n=21) videos of frontal and lateral hand gestures of five-seconds (e.g., raising a hand), followed by visually presented sentences that are either congruent or incongruent with the hand gesture (e.g., 'the mountain is high/low…'). All participants underwent a semantic-probe task, judging whether a target word is related or unrelated to the gesture-speech event. EEG results suggest that, during the perception phase of handgestures, while both frontal and lateral gestures elicited power decrease in both the alpha (8-12Hz) and the beta (16-24Hz) bands, lateral gestures elicited reduced power decrease in the beta band when compared with frontal gestures. For sentence comprehension, at the critical word whose meaning is congruent/incongruent with the gesture, frontal gestures elicited an N400 effect for gesture-sentence incongruency. More importantly, this incongruency effect was significantly reduced for lateral gestures. The findings suggest that body orientation plays a crucial role in gesture perception, and that its inferred socialcommunicative intention influences gesture-sentence semantic integration in an interactive manner.

show abstract

“…These frequency bands are delta (1-4 Hz), theta (4-7 Hz), alpha (8-12 Hz) and low beta (13-19 Hz). Overall, alpha and low beta desynchronization have been shown to be associated with retrieval of lexical and semantic information (Bakker et al, 2015;Bastiaansen et al, 2008;Berger et al, 2014;He et al, 2015;Kielar et al, 2014;Li and Yang, 2013;Shahin et al, 2009;Strauß et al, 2014;Willems et al, 2008). Theta synchronization is linked to memory processes involved in lexical and semantic processing as well as in working memory and executive functions during memory retrieval (Bastiaansen et al, 2008;Bakker et al, 2015;Ketz et al, 2014;Li and Yang, 2013;Maguire et al, 2010;Shahin et al, 2009;Strauß et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

Common and differential electrophysiological mechanisms underlying semantic object memory retrieval probed by features presented in different stimulus types

Chiang

Eroh

Spence

et al. 2016

International Journal of Psychophysiology

View full text Add to dashboard Cite

The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech

Cited by 39 publications

References 81 publications

Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving

Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving

Gesture’s body orientation modulates the N400 during semantic integration of gesture and visual sentence

Common and differential electrophysiological mechanisms underlying semantic object memory retrieval probed by features presented in different stimulus types

Contact Info

Product

Resources

About