Making sense of objects lying around: How contextual objects shape brain activity during action observation

El-Sourani, Nadiya; Wurm, Moritz F.; Trempler, Ima; Fink, Gereon R.; Schubotz, Ricarda Ines

doi:10.1016/j.neuroimage.2017.11.047

Cited by 18 publications

(24 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike the omnipresence of ATL hub, the spoke regions participated in a content-related fashion: In Experiment 1, extensive clusters of the frontoparietal action system (the pre- and post-central gyri, inferior frontal gyrus, and parietal lobules) were modulated by semantic congruity about actions. These areas correspond to the typical lesion sites seen in apraxia patients (Binkofski and Buxbaum, 2013) and the voluminous fMRI literature about motor imagery and action knowledge (e.g., El-Sourani et al., 2018; Hétu et al., 2013; van Elk et al., 2014; Wurm and Lingnau, 2015). By comparison, in Experiment 2, two key regions of the medial place-related system, the LG and RSC, were recruited to process semantic congruency about places.…”

Section: Discussionsupporting

confidence: 69%

Unveiling the dynamic interplay between the hub- and spoke-components of the brain's semantic system and its impact on human behaviour

Chiou

Ralph

2019

NeuroImage

View full text Add to dashboard Cite

The neural architecture of semantic knowledge comprises two key structures: ( i ) A set of widely dispersed regions, located adjacent to the sensorimotor cortices, serve as spokes that represent various modality-specific and context-dependent contents. ( ii ) The anterior-temporal lobe (ATL) serves as a hub that computes the nonlinear mappings required to transform modality-specific information into pan-modality, multifaceted concepts. Little is understood regarding whether neural dynamics between the hub and spokes might flexibly alter depending on the nature of a concept and how it impinges upon behaviour. Using fMRI, we demonstrate for the first time that the ATL serves as a ‘pivot’ which dynamically forms flexible long-range networks with cortical modules specialised for different domains (in the present case, the knowledge about actions and places). In two experiments, we manipulated semantic congruity and asked participants to recognise visually presented items. In Experiment 1 (dual-object displays), the ATL increased its functional coupling with the bilateral frontoparietal action-sensitive system when the objects formed a pair that permitted semantically meaningful action. In Experiment 2 (objects embedded in a scene), the ATL augmented its coupling with the retrosplenial cortex of the place-sensitive system when the objects and scene formed a semantically coherent ensemble. Causative connectivity revealed that, while communication between the hub and spokes was bidirectional, the hub's directional impact on spokes dwarfed the strength of the inverse spoke-to-hub connectivity. Furthermore, the size of behavioural congruity effects co-varied with the strength of neural coupling between the ATL hub and action- / place-related spokes, evident both at the within -individual level (the behavioural fluctuation across scanning runs) and between -individual level (the behavioural variation of between participants). Together, these findings have important implications for understanding the machinery that links neural dynamics with semantic cognition.

show abstract

Section: Discussionsupporting

confidence: 69%

Unveiling the dynamic interplay between the hub- and spoke-components of the brain's semantic system and its impact on human behaviour

Chiou

Ralph

2019

NeuroImage

View full text Add to dashboard Cite

show abstract

“…Caspers, Zilles, Laird, & Eickhoff, 2010;van Overwalle & Beatens, 2009). To this end, action observers can exploit various sources of information, including not only moving body parts (i.e., manipulation movements) and manipulated objects but also various contextual factors, such as the room , the actor (Hrkać, Wurm, & Schubotz, 2014), additional objects in a scene (contextual objects [COs]; El-Sourani, Wurm, Trempler, Fink, & Schubotz, 2018), and spatial relations between objects and agents (Brozzoli, Gentile, Bergouignan, & Ehrsson, 2013;Costantini, Committeri, & Sinigaglia, 2011). Although the influence of contextual information on object recognition has been intensively investigated (Barenholtz, 2014;Zimmermann, Schnier, & Lappe, 2010;Hayes, Nadel, & Ryan, 2007;Bar, 2004;Boyce, Pollatsek, & Rayner, 1989), its impact on action understanding has so far been addressed by only a few studies.…”

Section: Introductionmentioning

confidence: 99%

Predictive Impact of Contextual Objects during Action Observation: Evidence from Functional Magnetic Resonance Imaging

El-Sourani

Trempler

Wurm

et al. 2020

Journal of Cognitive Neuroscience

Self Cite

View full text Add to dashboard Cite

The processing of congruent stimuli, such as an object or action in its typical location, is usually associated with reduced neural activity, probably due to facilitated recognition. However, in some situations, congruency increases neural activity—for example, when objects next to observed actions are likely versus unlikely to be involved in forthcoming action steps. Here, we investigated using fMRI whether the processing of contextual cues during action perception is driven by their (in)congruency and, thus, informative value to make sense of an observed scene. Specifically, we tested whether both highly congruent contextual objects (COs), which strongly indicate a future action step, and highly incongruent COs, which require updating predictions about possible forthcoming action steps, provide more anticipatory information about the action course than moderately congruent COs. In line with our hypothesis that especially the inferior frontal gyrus (IFG) subserves the integration of the additional information into the predictive model of the action, we found highly congruent and incongruent COs to increase bilateral activity in action observation nodes, that is, the IFG, the occipitotemporal cortex, and the intraparietal sulcus. Intriguingly, BA 47 was significantly stronger engaged for incongruent COs reflecting the updating of prediction in response to conflicting information. Our findings imply that the IFG reflects the informative impact of COs on observed actions by using contextual information to supply and update the currently operating predictive model. In the case of an incongruent CO, this model has to be reconsidered and extended toward a new overarching action goal.

show abstract

“…Future studies have to examine how much real object or contextual information would further improve this performance level. Especially, object information provides an efficient restriction on to-be-expected manipulations [6][7][8][9][10][11]. It remains to be tested how non-spatial object information potentially interacts with the exploitation of static and dynamic spatial relations between objects involved in actions.…”

Section: Plos Onementioning

confidence: 99%

“…Though in recent years, some progress has been made towards computationally more concrete models of the mechanisms and processes underlying action recognition [5], it still remains largely unresolved how the brain accomplishes this complex task. Prediction of actions can rely on different sources of information, including manipulated objects [6][7][8][9], contextual objects [10,11], movements [12], context [13] and features regarding the actress or actor [14]. A major aim of ongoing research is to disentangle the respective contribution and relevance of these sources of information feeding human action prediction.…”

Section: Introductionmentioning

confidence: 99%

Using enriched semantic event chains to model human action prediction based on (minimal) spatial information

et al. 2020

Self Cite

View full text Add to dashboard Cite

Predicting other people’s upcoming action is key to successful social interactions. Previous studies have started to disentangle the various sources of information that action observers exploit, including objects, movements, contextual cues and features regarding the acting person’s identity. We here focus on the role of static and dynamic inter-object spatial relations that change during an action. We designed a virtual reality setup and tested recognition speed for ten different manipulation actions. Importantly, all objects had been abstracted by emulating them with cubes such that participants could not infer an action using object information. Instead, participants had to rely only on the limited information that comes from the changes in the spatial relations between the cubes. In spite of these constraints, participants were able to predict actions in, on average, less than 64% of the action’s duration. Furthermore, we employed a computational model, the so-called enriched Semantic Event Chain (eSEC), which incorporates the information of different types of spatial relations: (a) objects’ touching/untouching, (b) static spatial relations between objects and (c) dynamic spatial relations between objects during an action. Assuming the eSEC as an underlying model, we show, using information theoretical analysis, that humans mostly rely on a mixed-cue strategy when predicting actions. Machine-based action prediction is able to produce faster decisions based on individual cues. We argue that human strategy, though slower, may be particularly beneficial for prediction of natural and more complex actions with more variable or partial sources of information. Our findings contribute to the understanding of how individuals afford inferring observed actions’ goals even before full goal accomplishment, and may open new avenues for building robots for conflict-free human-robot cooperation.

show abstract

Making sense of objects lying around: How contextual objects shape brain activity during action observation

Cited by 18 publications

References 50 publications

Unveiling the dynamic interplay between the hub- and spoke-components of the brain's semantic system and its impact on human behaviour

Unveiling the dynamic interplay between the hub- and spoke-components of the brain's semantic system and its impact on human behaviour

Predictive Impact of Contextual Objects during Action Observation: Evidence from Functional Magnetic Resonance Imaging

Using enriched semantic event chains to model human action prediction based on (minimal) spatial information

Contact Info

Product

Resources

About