Rational use of cognitive resources in human planning

Abstract

A critical aspect of human intelligence is our ability to plan, that is, to use a model of the world to simulate, evaluate, and select among hypothetical future actions. However, exhaustive planning is intractable because the number of possible action sequences increases exponentially with the number of steps that one plans ahead. Understanding how people are nevertheless able to solve novel problems when their actions have long-reaching consequences is thus critical to understanding human intelligence. Progress in answering this question has been hampered by two challenges: planning cannot be observed and we do not have a good framework for formalizing the tradeoff between performance and computational cost. In this work, we propose solutions to both challenges, based on the idea that planning can be conceptualized as information seeking. Specifically, we model planning as the selection of information-generating computations and introduce an experimental paradigm in which these computations are externalized as mouse clicks. We find that our participants' behavior is broadly consistent with the optimal information-seeking model. We also uncover systematic deviations that might result from heuristic approximations or additional cognitive constraints that have yet to be uncovered.

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

“…These findings are consistent with an emerging new perspective that frames decision-making as a process of active information search rather than simply the passive accumulation of reward estimates (Hunt, 2021). This view not only bridges decision-making with active informationsampling (Boldt, Blundell, & De Martino, 2019;Cohen, McClure, & Yu, 2007;Gottlieb, 2018;Gottlieb, Cohanpour, Li, Singletary, & Zabeh, 2020;Gottlieb & Oudeyer, 2018;Hunt et al, 2018;Hunt, Rutledge, Malalasekera, Kennerley, & Dolan, 2016;Kaanders, Nili, O'Reilly, & Hunt, 2020), extended behaviors (Callaway, van Opheusden, et al, 2021;Holroyd & Yeung, 2012), and learning (Behrens, Woolrich, Walton, & Rushworth, 2007;Frömer et al, 2020;Nassar et al, 2012;O'Reilly, 2013), it also renders decision-making fundamentally a control problem. A group of recently proposed process models puts information search -rather than value comparison -at the core of the decision process (Callaway, Rangel, & Griffiths, 2021;Jang, Sharma, & Drugowitsch, 2021).…”

Decision Prioritization and Causal Reasoning in Decision Hierarchies

“…There are many differences between our study and previous planning studies in humans. Most of these studies relied on tasks without uncertainty (Classical planning) or tasks in which the uncertainty is limited to stochastic transitions between states (Markov Decision Processes, MDPs), and have focused on how people cope with the combinatorial explosion that occurs as the planning horizon increases (Keramati et al, 2016;Huys et al, 2012;Callaway et al, 2021), the depth with which people plan (Snider et al, 2015;van Opheusden et al, 2021), or the extent to which people use model-based or model-free strategies when learning from reinforcement (Daw et al, 2011(Daw et al, , 2005Keramati et al, 2016). The present study is different because we focus on how people disambiguate a single hidden state from a sequence of information-seeking and reward-seeking actions.…”

Leveraging Machine Learning to Automatically Derive Robust Decision Strategies from Imperfect Knowledge of the Real World

“…Given a model of the environment, the resource-rational heuristic h ⋆ for an agent with the computational resources B can be computed by reformulating the definition of the resource-rational heuristic as the solution to a metalevel Markov decision process (MDP) and applying methods from dynamic programming or reinforcement learning to compute its optimal policy (Callaway et al, 2018a;Krueger et al, 2022;Callaway et al, 2022b). This approach models the decision process as a series of computations that can be chosen one by one.…”

“…Recent work has built on the definition of optimal heuristics for human decisionmaking by (Lieder and Griffiths, 2020) to develop machine learning methods for discovering clever heuristics for human decision-making (Callaway et al, 2018a; Aashay Mehta and Yash Raj Jain contributed equally to this work. Gul et al, 2018;Krueger et al, 2022;Callaway et al, 2022b;Skirzyński et al, 2021;Consul et al, 2022) as well as intelligent cognitive tutors that teach them to people (Callaway et al, 2022a;Consul et al, 2022) and AI-generated decision aids that guide people through the application of the discovered strategies (Becker et al, 2022). Here, we use the term "heuristic" in the broad sense of "any decision strategy that uses only a subset of all potentially relevant information and is not guaranteed to always yield the optimal solution.…”

“…Unlike previous normative theories, such as expected utility theory (von Neumann and Morgenstern, 1944), it takes into account that people's time and cognitive resources are bounded. Its prescriptions for good decision-making (Lieder and Griffiths, 2020;Krueger et al, 2022;Callaway et al, 2022b) thus, at least sometimes, resemble simple fast-and-frugal heuristics (Gigerenzer & Todd, 1999).…”