The role of prediction and outcomes in adaptive cognitive control

Abstract

Humans adaptively perform actions to achieve their goals. This flexible behaviour requires two core abilities: the ability to anticipate the outcomes of candidate actions and the ability to select and implement actions in a goal-directed manner. The ability to predict outcomes has been extensively researched in reinforcement learning paradigms, but this work has often focused on simple actions that are not embedded in hierarchical and sequential structures that are characteristic of goal-directed human behaviour. On the other hand, the ability to select actions in accordance with high-level task goals, particularly in the presence of alternative responses and salient distractors, has been widely researched in cognitive control paradigms. Cognitive control research, however, has often paid less attention to the role of action outcomes. The present review attempts to bridge these accounts by proposing an outcome-guided mechanism for selection of extended actions. Our proposal builds on constructs from the hierarchical reinforcement learning literature, which emphasises the concept of reaching and evaluating informative states, i.e., states that constitute subgoals in complex actions. We develop an account of the neural mechanisms that allow outcome-guided action selection to be achieved in a network that relies on projections from cortical areas to the basal ganglia and back-projections from the basal ganglia to the cortex. These cortico-basal ganglia-thalamo-cortical 'loops' allow convergence - and thus integration - of information from non-adjacent cortical areas (for example between sensory and motor representations). This integration is essential in action sequences, for which achieving an anticipated sensory state signals the successful completion of an action. We further describe how projection pathways within the basal ganglia allow selection between representations, which may pertain to movements, actions, or extended action plans. The model lastly envisages a role for hierarchical projections from the striatum to dopaminergic midbrain areas that enable more rostral frontal areas to bias the selection of inputs from more posterior frontal areas via their respective representations in the basal ganglia.

Uncertainty and expectancy deviations require cortico-subcortical cooperation

“…Consistently with these results, the striatum is shown to have a particular role in overriding dominant and/or pre-biased responses, choosing appropriate responses among competing alternatives and suppressing unwanted ones, and switching from a planned or automatic response to an alternative or controlled one in light of a sudden, unexpected change (e.g. Cameron et al, 2009;Cools et al, 2006;den Ouden et al, 2010;Hikosaka and Isoda, 2010;Mestres-Misse et al, 2012, 2014aRedgrave et al, 1999;Schiffer et al, 2015). Consequently, the present results support the decisive function of the striatum in behavioral and cognitive control, flexibility, and adjustment.…”

“…We have previously demonstrated how the basal ganglia and corticostriatal systems substantially contribute to overriding habitual but currently incompatible behavior, and to engaging the most appropriate response given actual relevant information (MestresMisse et al, 2012(MestresMisse et al, , 2014acf. den Ouden et al, 2010;Redgrave et al, 1999;Schiffer et al, 2015).…”

Incorrect predictions reduce switch costs

“…Kunde, Elsner, & Kiesel, 2007;Marien, Aarts, & Custers, 2013) as well as on the basis of neurophysiological evidence suggesting an integral role for predictions in the control of behavior in general (cf. Schiffer, Waszak, & Yeung, 2015, for a recent review). The question arises whether asking participants to engage in certain predictions merely constraints and directs processes which proceed anyway but in a rather unsystematic manner, at least under conditions that are designed to allow for no valid predictions.…”

Special issue: Neural basis of adaptive control

“…-from different models: from rodents (Laubach et al, 2015;Newman et al, 2015;Lesaint et al, 2015;Warren et al, 2015), to monkeys (San- Galli and Bouret, 2015;O'Neill and Schultz, 2015;Stuphorn, 2015) to humans (Bonnelle et al, 2015;Cavanagh and Shackman, 2015;Kelly and O'Connell, 2015;Ridderinkhof and Brass, 2015;Schiffer et al, 2015) with computational models (Lesaint et al, 2015;Schiffer et al, 2015); -at different level of explanations, from lesion studies (Newman et al, 2015) to electrophysiological recordings at the population (Cavanagh and Shackman, 2015;Kelly and O'Connell, 2015;Laubach et al, 2015;Warren et al, 2015) and single cells levels (O'Neill and Schultz, 2015;Stuphorn, 2015).…”