People accurately predict the transition probabilities between actions

Thornton, Mark Allen; Tamir, Diana

doi:10.1126/sciadv.abd4995

Cited by 20 publications

(11 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our perception of others’ actions is both efficient and flexible, enabling us to rapidly understand new actions no matter where they occur or who is performing them. This understanding plays a part in complex social computations about the mental states and intentions of others ( Jamali et al, 2021 ; Spunt et al, 2011 ; Thornton et al, 2019 ; Thornton and Tamir, 2021 ; Weaverdyck et al, 2021 ). Visual action recognition also interacts cross-modally with language-based action understanding ( Bedny and Caramazza, 2011 ; Humphreys et al, 2013 ).…”

Section: Introductionmentioning

confidence: 99%

Social-affective features drive human representations of observed actions

Dima

Tomita

Honey

et al. 2022

eLife

View full text Add to dashboard Cite

Humans observe actions performed by others in many different visual and social settings. What features do we extract and attend when we view such complex scenes, and how are they processed in the brain? To answer these questions, we curated two large-scale sets of naturalistic videos of everyday actions and estimated their perceived similarity in two behavioral experiments. We normed and quantified a large range of visual, action-related, and social-affective features across the stimulus sets. Using a cross-validated variance partitioning analysis, we found that social-affective features predicted similarity judgments better than, and independently of, visual and action features in both behavioral experiments. Next, we conducted an electroencephalography experiment, which revealed a sustained correlation between neural responses to videos and their behavioral similarity. Visual, action, and social-affective features predicted neural patterns at early, intermediate, and late stages, respectively, during this behaviorally relevant time window. Together, these findings show that social-affective features are important for perceiving naturalistic actions and are extracted at the final stage of a temporal gradient in the brain.

show abstract

Section: Introductionmentioning

confidence: 99%

Social-affective features drive human representations of observed actions

Dima

Tomita

Honey

et al. 2022

eLife

View full text Add to dashboard Cite

show abstract

“…Returning to the example above, eating and cooking are close together on the food dimension in particular, so this proximity could allow perceivers to accurately predict that this likely transition. Indeed, recent behavioral and neural investigations have found evidence for precisely this hypothesis: proximity on the ACT-FAST dimensions mediates the accuracy of judgements about action transitions (Thornton & Tamir, 2020b, 2021). These findings demonstrate a novel mechanism for action prediction, as well as further evidence for the psychological usefulness of the ACT-FAST.…”

Section: Discussionmentioning

confidence: 97%

Six dimensions describe action understanding: The ACT-FASTaxonomy.

Thornton¹,

Tamir²

2022

Journal of Personality and Social Psychology

Self Cite

View full text Add to dashboard Cite

Humans engage in a wide variety of different actions and activities. These range from simple motor actions like reaching for an object, to complex activities like governing a nation.Navigating everyday life requires people to make sense of this diversity of actions. We suggest that the mind simplifies this complex domain by attending primarily to the most essential features of actions. Using a parsimonious set of action dimensions, the mind can organize action knowledge in a low-dimensional representational space. In seven studies, we derive and validate such an action taxonomy. Study 1 uses large-scale text analyses to generate and test potential action dimensions. Study 2 validates interpretable labels for these dimensions. Studies 3-5 demonstrate that these dimensions can explain human judgments about actions. We perform model selection on data from these studies to arrive at the optimal set of six psychological dimensions, together forming the Abstraction, Creation, Tradition, Food, Animacy, Spiritualism Taxonomy (ACT-FAST). Study 6 demonstrates that ACT-FAST can predict socially relevant qualities of actions, including how, when, where, why, and by whom they are performed.Finally, Study 7 shows that ACT-FAST can explain action-related patterns of brain activity using naturalistic fMRI. Together, these studies reveal the dimensional structure the mind applies to organize action concepts.

show abstract

“…To date there has been only limited data-driven work in the action domain. Using principal component analysis (PCA) of large-scale text data, a low-dimensional taxonomy of actions has been shown to explain neural data and human action judgments (Thornton & Tamir, 2021a), as well as guide predictions about actions (Thornton & Tamir, 2021b). However, since this taxonomy was generated from text data, most of these dimensions were relatively abstract (e.g.…”

Section: Introductionmentioning

confidence: 99%

A data-driven investigation of human action representations

Dima

Hebart

Işık

2022

Preprint

View full text Add to dashboard Cite

Understanding actions performed by others requires us to integrate different types of information about people, scenes, objects, and their interactions. What organizing dimensions does the mind use to make sense of this complex action space? To address this question, we collected intuitive similarity judgments across two large-scale sets of naturalistic videos depicting everyday actions. We used cross-validated sparse non-negative matrix factorization (NMF) to identify the structure underlying action similarity judgments. A low-dimensional representation, consisting of nine to ten dimensions, was sufficient to accurately reconstruct human similarity judgments. The dimensions were robust to stimulus set perturbations and reproducible in a separate odd-one-out experiment. Human labels mapped these dimensions onto semantic axes relating to food, work, and home life; social axes relating to people and emotions; and one visual axis related to scene setting. While highly interpretable, these dimensions did not share a clear one-to-one correspondence with prior hypotheses of action-relevant dimensions. Together, our results reveal a low-dimensional set of robust and interpretable dimensions that organize intuitive action similarity judgments and highlight the importance of data-driven investigations of behavioral representations.

show abstract

People accurately predict the transition probabilities between actions

Cited by 20 publications

References 49 publications

Social-affective features drive human representations of observed actions

Social-affective features drive human representations of observed actions

Six dimensions describe action understanding: The ACT-FASTaxonomy.

A data-driven investigation of human action representations

Contact Info

Product

Resources

About