Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex

Abstract

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available—a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.

Brain Substrates of Recovery from Misleading Influence

“…Our results tie together past findings in primates and humans demonstrating that the alPFC (sometimes referred to as rostrolateral PFC, lateral orbitofrontal cortex, frontopolar cortex, or ventrolateral PFC, all with similar spatial coordinates, e.g., Clark et al, 2004;Mendelsohn et al, 2008;Bunge et al, 2009;Boorman et al, 2011;Sakaki et al, 2011;Badre et al, 2012) possesses the necessary capabilities to partake of restorative processes and represent the value of switching to a counterfactual or alternative choice (Cabeza and Nyberg, 1997;Ramnani and Owen, 2004;Koechlin and Hyafil, 2007;Mendelsohn et al, 2008;Bunge et al, 2009;Rushworth et al, 2011;Sakaki et al, 2011;Badre et al, 2012).…”

Social norms, self-control, and the value of antisocial behavior

“…magnitude, probability, delay, uncertainty, and effort) [33][34][35]. This integrated signal may be incorporated into a process of episodic prospection that simulates outcomes for an array of action options, permitting a comparison of expected value for multiple potential actions [36][37][38][39]. Finally, DLPFC appears to be important for representing associations between abstract rules and expected outcome values [40].…”

Counterfactual thinking affects the excitability of the motor cortex

“…In the context of human decision-making, representation of value of choices that are taken plays an essential role in guiding choice behaviour, but there is also a considerable adaptive advantage in representing the potential value of choices that are untaken (Boorman, Behrens, & Rushworth, 2011). When faced with mutually exclusive options, the choice we make is conditioned not only by what we hope to gain, but also by how we hope we will feel afterward (Camille et al, 2004).…”