Explicit knowledge of task structure is the primary determinant of human model-based action

Abstract

Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a novel variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and individuals suffering from obsessive-compulsive (OCD) or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of subjects, and less in OCD. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, explicit task structural knowledge determines human use of model-based reinforcement learning, and is most readily acquired from instruction rather than experience .

Problem-solving prior to instruction in learning motor skills - initial self-determined practice improves javelin throwing performance

“…They showed that an increase in explicit processes siphons away resources needed by implicit processes for learning. This suppression of implicit processes due to the dominance of explicit processes has also recently been observed in the context of decision making (Castro-Rodrigues et al, 2022). Applying these recent findings to the current study, this could mean that explicit processes become dominant when instruction comes first, and stay dominant even when subjects enter the self-determined practice phase.…”

The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection

“…However, when the actions necessitated by each second-step states are consistent from trial to trial, reaction time differences may reflect preparatory activity at the motor level, on the basis of correlation between first-step choice and the action that will be required at the second step. Indeed, recent studies in humans have demonstrated that motor responses can show sensitivity to task structure when choices are model free ( Castro-Rodrigues et al., 2020 ; Konovalov and Krajbich, 2020 ). Therefore in versions of the task, including ours, that do not randomize the action associated with each second-step option from trial to trial (as done in the original human task but not in rodent versions), second-step reaction times may not provide strong evidence for model-based action evaluation.…”

Spontaneous instrumental avoidance learning in social contexts

“…Interestingly, explicit task structural knowledge has been shown to increase participants use of model-based, rather than model-free, reinforcement learning (Castro-Rodrigues et al, 2020). In the present task, subjects with an explicit strategy did efficiently integrate counterfactual information, thus behaving more similarly to what would be expected from an instructed group.…”