Theory of Mind and Joint Attention

Abstract

Published as part ofThe Handbook on Socially Interactive Agents Chapter 9 -Theory of Mind and Joint Attention environment, or coordinate interactions, such as biological motion, facial expressions, and eye and head movements. Furthermore, those signals are tightly linked to higher cognitive processes like recognizing others' feelings and internal states, identifying others' intentions, or deciding whether they are friend or foe. Understanding the neurobiological basis of behavior is crucial for cognitive, developmental, clinical, comparative, and social psychology, as well as philosophy and evolutionary anthropology [Singer 2012]. In this context, examining the engagement of social brain areas in interactions with artificial agents seems like a natural step to follow for HAI as well." Using a combination of subjective/explicit measures like self-report and questionnaires, and objective measures like metrics related to performance, behavior, psychophysiology, and neuroimaging, HAI is able to have a more comprehensive view of behavioral and brain mechanisms involved in social interactions with human and non-human agents. The present chapter provides an overview from the perspective of SCN regarding theory of mind (ToM) and joint attention (JA) as crucial mechanisms of social cognition and discusses how these mechanisms have been investigated in social interaction with artificial agents. In the final sections, the chapter reviews computational models of ToM and JA in social robots (SRs) and intelligent virtual agents (IVAs) and discusses the current challenges and future directions. Theory of Mind and Joint Attention-Crucial Mechanisms of Social Cognition Theory of MindImagine that as you are walking on the street someone is coming from the opposite direction toward you. At some point, and before passing each other, the person all of a sudden puts the palm of her/his hand on their forehead, stops walking, turns around, and goes back to where she/he came from. How would you explain this behavior? Probably, you would guess that she/he might have forgotten something and decided to go back. And you might be right. Importantly, most of the explanations you would choose to explain the observed behavior would refer to mental states, such as thoughts, preferences, intentions, or emotions. This is based on the ability to perceive and understand that others have beliefs, desires, goals, and knowledge different from your

Can the robot “see” what I see? Robot gaze drives attention depending on mental state attribution

“…The adoption of the intentional stance (Dennett, 1989) refers to the adoption of the strategy in explaining the behavior of artifacts, whether they are cartoons, virtual agents, schematic faces, or robots, in terms of having mental states. Mentalizing refers to inferring these mental states in a specific context (Perez-Osorio et al, 2021). Notably, adopting the intentional stance and mentalizing are closely related, with adopting the intentional stance possibly being required to make mental state attributions and vice versa.…”

“…This dual function of gaze allows us to look at a point in space and convey a message to a partner so they, too, look at the same point, either reflexively or–as has become clear recently–more strategically, depending on several social factors (Dalmaso et al, 2020 ). This action would initiate a joint attention process (Tomasello, 1995 ), a precursor and facilitator of the attribution of mental states (Perez-Osorio et al, 2021 ).…”

Social Perception Deficit as a Factor of Vulnerability to Psychosis: A Brief Proposal for a Definition

“…Humans can rapidly perceive the emotions experienced by others (emotional perception; Jang and Elfenbein, 2015 ) and use them, together with other sources of information, to grasp their internal reasoning ( Castelli, 2015 ). Some suggest that this process can take the form of a sequence where perceptual inputs from sensory stimuli are used to generate inferential interpretations of others’ minds ( Goldman, 2006 ; Pérez-Osorio et al, 2021 ). Others suggest that we can also perceive directly (rather than inferring ) other people’s intentions ( Dokic, 2010 ; Kiverstein, 2015 ; Gallagher, 2017 ).…”