Mind the gap: Challenges of deep learning approaches to Theory of Mind

Aru, Jaan; Labash, Aqeel; Corcoll, Oriol; Vicente, Raúl

doi:10.48550/arxiv.2203.16540

Cited by 1 publication

(3 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a wealth of literature about modeling other agents in multi-agent systems using classical methods [6], but we specifically focus on the use of deep neural networks for the task [2]. Furthermore, while most of the work in the area of ToM for artificial agents focuses on evaluating their beliefs-for example, if they pass some variation of the Sally-Anne test [7,8]; then, we specifically focus on the intentions of the agents.…”

Section: Related Workmentioning

confidence: 99%

“…Studying human-level intentions (and ToM) has proven to be complicated [1,[28][29][30]; studying simple tasks with agents whose representations can be examined will provide unique insights into the emergence of more complex aspects of ToM [2]. In particular, being able to manipulate the network architecture and the components of the system allows one to answer which aspects really matter for solving a particular task.…”

Section: Reading Out Intentionsmentioning

confidence: 99%

“…The ability of humans to infer the mental states of others, such as their beliefs, desires, or intentions, is called theory of mind (ToM) [1,2]. Inferring other agents' intentions gives an advantage both in cooperative tasks, where participants have to coordinate their activities, and in competitive tasks, where one might want to guess the next move of one's opponent.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Do Deep Reinforcement Learning Agents Model Intentions?

Matiisen

Labash

Majoral

et al. 2022

Stats

View full text Add to dashboard Cite

Inferring other agents’ mental states, such as their knowledge, beliefs and intentions, is thought to be essential for effective interactions with other agents. Recently, multi-agent systems trained via deep reinforcement learning have been shown to succeed in solving various tasks. Still, how each agent models or represents other agents in their environment remains unclear. In this work, we test whether deep reinforcement learning agents trained with the multi-agent deep deterministic policy gradient (MADDPG) algorithm explicitly represent other agents’ intentions (their specific aims or plans) during a task in which the agents have to coordinate the covering of different spots in a 2D environment. In particular, we tracked over time the performance of a linear decoder trained to predict the final targets of all agents from the hidden-layer activations of each agent’s neural network controller. We observed that the hidden layers of agents represented explicit information about other agents’ intentions, i.e., the target landmark the other agent ended up covering. We also performed a series of experiments in which some agents were replaced by others with fixed targets to test the levels of generalization of the trained agents. We noticed that during the training phase, the agents developed a preference for each landmark, which hindered generalization. To alleviate the above problem, we evaluated simple changes to the MADDPG training algorithm which lead to better generalization against unseen agents. Our method for confirming intention modeling in deep learning agents is simple to implement and can be used to improve the generalization of multi-agent systems in fields such as robotics, autonomous vehicles and smart cities.

show abstract

Section: Related Workmentioning

confidence: 99%