Formalizing emotion concepts within a Bayesian model of theory of mind

Saxe, Rebecca; Houlihan, Sean Dae

doi:10.1016/j.copsyc.2017.04.019

Cited by 76 publications

(69 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Identity recognition enables us to acquire knowledge about specific individuals that we can retrieve in future encounters (5,31). Expression recognition helps us to infer the emotional states of an individual (50,51,44) and predict their future actions and reactions.…”

Section: Introductionmentioning

confidence: 99%

Recognition of identity and expressions as integrated processes

O'Nell¹,

Saxe²,

Anzellotti³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

According to the dominant account of face processing, recognition of emotional expressions is implemented by the superior temporal sulcus (STS), while recognition of face identity is implemented by inferior temporal cortex (IT) (Haxby et al., 2000). However, recent patient and imaging studies (Fox et al., 2011, Anzellotti et al. 2017) found that the STS also encodes information about identity.Jointly representing expression and identity might be computationally advantageous: learning to recognize expressions could lead to the emergence of representations that support identity recognition. To test this hypothesis, we trained a deep densely connected convolutional network (DenseNet, Huang et al., 2017) to classify face images from the fer2013 dataset as either angry, disgusted, afraid, happy, sad, surprised, or neutral. We then froze the weights of the DenseNet and trained linear layers attached to progressively deeper layers of this net to classify either emotion or identity using a subset of the Karolinska (KDEF) dataset. Finally, we tested emotion and identity classification in left out images in the KDEF dataset that were not used for training.Classification accuracy for emotions in the KDEF dataset increased from early to late layers of the DenseNet, indicating successful transfer across datasets. Critically, classification accuracy for identity also increased from early to late layers of this DenseNet, despite the fact that it had not been trained to classify identity. A linear layer trained on the DenseNet features vastly outperformed a linear layer trained on pixels (98.8% vs 68.7%), demonstrating that the high accuracy obtained with the DenseNet features cannot be explained by low-level confounds. These results show that learning to recognize facial expressions can lead to the spontaneous emergence of representations that support the recognition of identity, thus offering a principled computational account for the discovery of expression and identity representations within the same portion of STS.

show abstract

Section: Introductionmentioning

confidence: 99%

Recognition of identity and expressions as integrated processes

O'Nell¹,

Saxe²,

Anzellotti³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Together these results suggest that by five years of age, children already have an abstract, theory-like causal model of others' emotions: They understand how external events, actions, and mental states can together give rise to different affective states, and they can make systematic predictions about how others might feel based on available evidence (Gopnik & Wellman, 1992;Jara-Ettinger et al, 2016;Ong et al, in press;Saxe & Houlihan, 2017;Wellman & Gelman, 1992). The current study adds to the growing body of work on children's understanding of the link between beliefs and emotions (e.g., Bradmetz & Schneider, 1999;Doan et al, 2018;Hadwin & Perner, 1991;Harris et al, 1989;Lagattuta, 2014;Lagattuta et al, 1997;Lara et al, 2017;Ronfard & Harris, 2014;Wellman & Liu, 2004;, and provides the earliest evidence for expectation-based emotion reasoning (see Doan et al, 2018;Lara et al, 2017).…”

Section: Discussionmentioning

confidence: 89%

Integrating expectations and outcomes: Preschoolers’ developing ability to reason about others’ emotions

Asaba¹,

Ong²,

Gweon³

2018

Preprint

View full text Add to dashboard Cite

People's emotional experiences depend not only on what actually happened, but also on what they thought would happen. Inferring others' expectations and how these expectations influence others' emotions is an important aspect of human social intelligence, allowing us to understand how others feel even when outcomes are uninformative and expectations are not explicitly communicated. Prior work suggests an abstract understanding of how expectations modulate emotional responses may not emerge until 7 to 8 years of age. Using a novel paradigm that capitalizes on intuitive physics to generate contextually plausible expectations, we sought to find earlier evidence for expectation-based emotion inference. Given two bowlers who experienced identical outcomes (hitting 3 out of 6 pins), we varied the physical trajectory of their balls such that one would initially expect to hit all pins (High-Expectation), while the other would expect to hit nothing (Low-Expectation). In Experiment 1, while both 4- and 5-year-olds appropriately adjusted the characters' happiness ratings upward (Low-Expectation) or downward (High-Expectation) with respect to their initial emotions, only 5-year-olds made adjustments robust enough to manifest as higher happiness ratings for the Low-Expectation than High-Expectation character. We replicate these results in Experiments 2-3 and show that 5-year-olds reliably differentiate the characters' emotions even when explicit information about their expectations is unavailable and must be inferred from the physical context (i.e., ball trajectories). Together, these results provide the earliest evidence for expectation-based emotion reasoning and suggest that the ability to spontaneously generate and consider others' expectations continues to develop during preschool years.

show abstract

“…. In other words, how people reason from outcomes to emotion, or infer emotion from expressions, can all be modeled with the same domain‐general inference machinery, under a common Bayesian “Theory of Emotion” (Ong, Zaki, et al., ; Saxe & Houlihan, ).…”

Section: Discussionmentioning

confidence: 99%

“…Throughout the paper, we have posed affective cognition as a "computational-level" problem (Marr, 1982) and have focused on reviewing probabilistic approaches, which offer a natural solution to such inferential problems (e.g., Griffiths, Kemp, & Tenenbaum, 2008;Oaksford & Chater, 2007). The Bayesian approaches in the studies described here (de Melo et al, 2014;Saxe & Houlihan, 2017;Wu et al, 2018) share more commonalities than differences, and what differences exist lie mainly in which variables the different sets of authors chose to prioritize or simplify out. Indeed, Fig.…”

Section: Discussionmentioning

confidence: 99%

“…This intuitive understanding of how event outcomes and mental states impact emotions emerges early in life (Gross & Ballif, 1991;Harris, 1989;Lagattuta, Wellman, & Flavell, 1997;Repacholi, Meltzoff, Hennings, & Ruba, 2016;Wu, Muentener, & Schulz, 2017): Toddlers and infants understand that an agent's display of happiness reflects a satisfied desire, while an agent's display of sadness reflects a thwarted goal (Repacholi & Gopnik, 1997;Skerry & Spelke, 2014;Wellman, Phillips, & Rodriguez, 2000). By preschool, children associate a surprised emotional expression with a mismatch , Saxe and Houlihan (2017), and Wu et al (2018). We use standard graphical model notation: Shaded circles represent observable variables, whereas unshaded circles represent latent variables.…”

Section: Intuitive Causes Of Emotionsmentioning

confidence: 99%

See 1 more Smart Citation

Computational Models of Emotion Inference in Theory of Mind: A Review and Roadmap

Ong

Zaki

Goodman

2018

Topics in Cognitive Science

View full text Add to dashboard Cite

Research on social cognition has fruitfully applied computational modeling approaches to explain how observers understand and reason about others' mental states. By contrast, there has been less work on modeling observers' understanding of emotional states. We propose an intuitive theory framework to studying affective cognition-how humans reason about emotions-and derive a taxonomy of inferences within affective cognition. Using this taxonomy, we review formal computational modeling work on such inferences, including causal reasoning about how others react to events, reasoning about unseen causes of emotions, reasoning with multiple cues, as well as reasoning from emotions to other mental states. In addition, we provide a roadmap for future research by charting out inferences-such as hypothetical and counterfactual reasoning about emotions-that are ripe for future computational modeling work. This framework proposes unifying these various types of reasoning as Bayesian inference within a common "intuitive Theory of Emotion." Finally, we end with a discussion of important theoretical and methodological challenges that lie ahead in modeling affective cognition.

show abstract

Formalizing emotion concepts within a Bayesian model of theory of mind

Cited by 76 publications

References 69 publications

Recognition of identity and expressions as integrated processes

Recognition of identity and expressions as integrated processes

Integrating expectations and outcomes: Preschoolers’ developing ability to reason about others’ emotions

Computational Models of Emotion Inference in Theory of Mind: A Review and Roadmap

Contact Info

Product

Resources

About