Towards machines that understand people

Abstract

The ability to estimate the state of a human partner is an insufficient basis on which to build cooperative agents. Also needed is an ability to predict how people adapt their behavior in response to an agent's actions. We propose a new approach based on computational rationality, which models humans based on the idea that predictions can be derived by calculating policies that are approximately optimal given human‐like bounds. Computational rationality brings together reinforcement learning and cognitive modeling in pursuit of this goal, facilitating machine understanding of humans.

A Workflow for building Computationally Rational Models of Human Behavior

“…While systems that comply with this steering may exhibit stronger interaction performance (Colella et al, 2020), a more advanced system could aim to identify and learn from users' mental models of AI, their refinement over the course of the interaction, and the influence of mutable user goals on interaction behavior. Developing such mental models of AI systems is currently a research challenge, even more so in the context of learning these online during interaction (Howes et al, 2023;Steyvers and Kumar, 2022;Bansal et al, 2019). Co-operative multi-agent setups (C ¸elikok et al, 2019) with the user and the AI system as interacting agents are a promising approach to improve interactive behavior by better anticipating the user and their strategies -doing so with computationally rational user models would be an interesting avenue for further research (Howes et al, 2023); these are bound to be confronted by computational challenges in a real-time and interactive setting.…”

“…Developing such mental models of AI systems is currently a research challenge, even more so in the context of learning these online during interaction (Howes et al, 2023;Steyvers and Kumar, 2022;Bansal et al, 2019). Co-operative multi-agent setups (C ¸elikok et al, 2019) with the user and the AI system as interacting agents are a promising approach to improve interactive behavior by better anticipating the user and their strategies -doing so with computationally rational user models would be an interesting avenue for further research (Howes et al, 2023); these are bound to be confronted by computational challenges in a real-time and interactive setting. Computational efficiency and approximation to computationally rational behavior can however be achieved by employing surrogate computational rationality models using methods such as amortized inference (Moon et al, 2023) or likelihood-free inference (Aushev et al, 2023;Palestro et al, 2018;Hartig et al, 2011).…”

“…One obstacle involves difficulties in capturing the extensive adaptivity that characterizes real-world human behavior (Oulasvirta et al, 2022). To be practically applied, models need to account for how behavior changes as a function of beliefs, capabilities, goals, and the environment, which however is further complicated by the continual learning, adaptation, and exploration of individuals in their environment (Howes et al, 2023). Successfully tackling this challenge would advance efforts towards building computing systems that better understand people.…”

“…Computational rationality is a theory that has recently emerged as a candidate to address this challenge (Lewis et al, 2014;Gershman et al, 2015;Lieder and Griffiths, 2020;Oulasvirta et al, 2022;Howes et al, 2023) 1 It draws together ideas from cognitive sciences and machine learning to predict adaptive behavior in artificial and biological agents. Given our focus on human cognition in this article, the core assumption made by the theory is that our choices and behavior aim to maximize expected utility, however we are limited in that by bounds imposed by the environment (e.g., limited time within which to act), our cognition (e.g., limits of memory, attention, perception), and our bodies (e.g., biomechanics).…”