Decoding Actions at Different Levels of Abstraction

Abstract

Brain regions that mediate action understanding must contain representations that are action specific and at the same time tolerate a wide range of perceptual variance. Whereas progress has been made in understanding such generalization mechanisms in the object domain, the neural mechanisms to conceptualize actions remain unknown. In particular, there is ongoing dissent between motor-centric and cognitive accounts whether premotor cortex or brain regions in closer relation to perceptual systems, i.e., lateral occipitotemporal cortex, contain neural populations with such mapping properties. To date, it is unclear to which degree action-specific representations in these brain regions generalize from concrete action instantiations to abstract action concepts. However, such information would be crucial to differentiate between motor and cognitive theories. Using ROI-based and searchlight-based fMRI multivoxel pattern decoding, we sought brain regions in human cortex that manage the balancing act between specificity and generality. We investigated a concrete level that distinguishes actions based on perceptual features (e.g., opening vs closing a specific bottle), an intermediate level that generalizes across movement kinematics and specific objects involved in the action (e.g., opening different bottles with cork or screw cap), and an abstract level that additionally generalizes across object category (e.g., opening bottles or boxes). We demonstrate that the inferior parietal and occipitotemporal cortex code actions at abstract levels whereas the premotor cortex codes actions at the concrete level only. Hence, occipitotemporal, but not premotor, regions fulfill the necessary criteria for action understanding. This result is compatible with cognitive theories but strongly undermines motor theories of action understanding.

Feature-coding transitions to conjunction-coding with progression through human visual cortex

“…In sum, our novel method permits the systematic investigation of feature-and conjunction-coding, and may be applicable not just to vision but to other modalities such as audition or motor action (59,60). Within vision, the method enables future investigation of a range of features not included in our stimulus set (including color, orientation and motion), in order to examine how conjunctioncoding emerges for different feature types and combinations.…”

Dissecting the visual perception of body shape with the Garner selective attention paradigm

“…static/dynamic; verbal/visual) spans the lateral and ventral occipitotemporal cortex (e.g. Hafri et al, 2017;O'Toole et al, 2014;Wurm & Lingnau, 2015; for review see Lingnau & Downing, 2015).…”

Neural bases of action abstraction

“…This view is closely tied to the mirror neuron theory of embodied action understanding (Gallese, 2013; Rizzolatti & Sinigaglia, 2010) according to which similar neural systems are active whether an action is observed or performed. Other theories of action understanding hold that action simulation is not a primary mode of action understanding, but rather that actions are processed cognitively—i.e., that they are categorized and accessed without reliance on the motor system (Wurm, Ariani, Greenlee, & Lingnau, 2015; Wurm & Lingnau, 2015). …”

“…In this cognitive action semantics framework, posterior regions near the visual system, such as the lateral occipital cortex (LOC), and the posterior middle temporal gyrus (pMTG), along with inferior parietal regions are generally considered to be hubs of action representation (Leshinskaya & Caramazza, 2015; Wurm & Lingnau, 2015). For example, the inferior posterior parietal cortex (Binder et al, 2009) and the MTG (Bedny, Caramazza, Grossman, Pascual-Leone, & Saxe, 2008; Wu, Morganti, & Chatterjee, 2008) are associated with conceptual action associations, while the IFG (Thompson-Schill et al, 1998) and anterior temporal lobes are associated with domain-general semantic processing (Abel et al, 2015).…”