Monte Carlo Planning Method Estimates Planning Horizons during Interactive Social Exchange

Abstract

Reciprocating interactions represent a central feature of all human exchanges. They have been the target of various recent experiments, with healthy participants and psychiatric populations engaging as dyads in multi-round exchanges such as a repeated trust task. Behaviour in such exchanges involves complexities related to each agent’s preference for equity with their partner, beliefs about the partner’s appetite for equity, beliefs about the partner’s model of their partner, and so on. Agents may also plan different numbers of steps into the future. Providing a computationally precise account of the behaviour is an essential step towards understanding what underlies choices. A natural framework for this is that of an interactive partially observable Markov decision process (IPOMDP). However, the various complexities make IPOMDPs inordinately computationally challenging. Here, we show how to approximate the solution for the multi-round trust task using a variant of the Monte-Carlo tree search algorithm. We demonstrate that the algorithm is efficient and effective, and therefore can be used to invert observations of behavioural choices. We use generated behaviour to elucidate the richness and sophistication of interactive inference.

Humans Use Forward Thinking to Exert Social Control

“…Given our results, it is compelling to design tasks that focus on the way that subjects learn the model (in our terms, acquiring a value for the parameter ߜ ) in early trials or build complex models of their partners' minds (as in a cognitive hierarchy (Camerer et al, 2004)). Indeed, even though, in our task, the straightforward model based on norm-adjustment characterized participants' behavior well, there are more sophisticated alternatives that are used to characterize interpersonal interactions, such as the framework of interactive partiallyobservable Markov decision processes (Gmytrasiewicz and Doshi, 2005;Hula et al, 2015;Xiang et al, 2012). These might provide additional insights into the sorts of probing that our subjects presumably attempted in early trials to gauge controllability (and the ways this differs in both In and No Control conditions between subjects who do and do not suffer from substantial illusions of control).…”

Bayesian Inference of Other Minds Explains Human Choices in Group Decision Making

“…The interpretability of the POMDP framework offers an opportunity to study the neurocognitive mechanisms of group decision making in healthy and diseased brains. POMDPs and similar Bayesian models have previously proved useful in understanding neural responses in sensory decision making (Rao, 2010;Huang et al, 2012;Huang and Rao, 2013;Khalvati and Rao, 2015) and in tasks involving interactions with a single individual (Xiang et al, 2012;Yoshida et al, 2010;Hula et al, 2015;Baker et al, 2017). We believe the POMDP model we have proposed can likewise prove useful in interpreting neural responses and data from neuroimaging studies of group decision making tasks.…”

“…If such a multi-level theory of mind is extended to infinite depth, the game converges to the Nash equilibrium (Nash, 1950). In reality, however, such an infinite-depth theory of mind appears not to occur in actual social interactions among humans (Kagel and Roth, 2016;Camerer, 2011;Henrich et al, 2005), with multi-level theory of mind limited to very few levels as observed in some experiments (Camerer et al, 2004;Yoshida et al, 2008Yoshida et al, , 2010Hula et al, 2015). Moreover, we expect less levels of theory of mind in group decision making in general compared to two-person interactions due to the complexity of the group decision making process and the anonymity of individuals.…”

“…Several studies have presented evidence in favor of the model-based approach by quantifying the similarity between probabilistic model-based methods and human behavior when the subject interacts with or reasons about another human (Ray et al, 2009;Yoshida et al, 2010;Xiang et al, 2012;Hula et al, 2015;Baker et al, 2017). Additional evidence in favor of a model-based approach comes from studies suggesting that ToM, autobiographical memory, and thinking about the future are all supported by a network of interconnected nodes in the brain known as the Default Mode Network (DMN) (Spreng et al, 2009;Spreng and Grady, 2010).…”

The shift from life in water to life on land advantaged planning in visually-guided behavior

“…While the selective benefit of planning is dependent on habitat, we expect there is no dependence of our results on our choice to only model planning in the simulated prey, versus allowing the predator to plan. This choice was based on the high computational burden of having both the prey and predator engaged in planning 38 . In addition, while most animals are prey (including rodents, where the leading non-primate model of planning, vicarious trial and error 11 , occurs), only a subset are predators, so this choice enables our results to have wider applicability.…”