Explicit knowledge of task structure is a primary determinant of human model-based action

Abstract

Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a novel variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and individuals suffering from obsessive-compulsive (OCD) or other disorders. Initial behaviour was modelfree, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of subjects, and less in OCD.Providing task structure information strongly increased model-based control, similarly across all groups.Thus, explicit task structural knowledge determines human use of model-based reinforcement learning, and is most readily acquired from instruction rather than experience..

Functional neuroimaging in psychiatry and the case for failing better

“…Firstly, the validity of model-derived findings is conditional on a correspondence between a hypothesized generative model and ''ground truth'' neurocognitive processes engaged by the task and, in many cases, an assumption of homogeneity in these processes both within and across participants (Bennett et al, 2019;Wilson and Collins, 2019). Yet, behavior even in simple tasks can ll Neuron 110, August 17, 2022 2527 reflect contributions from multiple cognitive processes, which can vary over time and differ between participants with the same diagnosis (Ashwood et al, 2022;Castro-Rodrigues et al, 2022;Collins and Frank, 2012;Collins et al, 2014Collins et al, , 2016Feher da Silva and Hare, 2020;Roy et al, 2020;Schlagenhauf et al, 2014). Thus, disentangling the relative contributions of distinct latent processes within and between participants requires carefully crafted tasks and detailed model comparison.…”

Using smartphones to optimise and scale-up the assessment of model-based planning

“…We demonstrated that estimates derived from the game are valid by replicating previously established correlates of modelbased planning: age (Eppinger et al, 2013), gender (Gillan et al, 2016), education (similar to previous work on IQ (Gillan et al, 2016) and processing speed (Schad et al, 2014)) and compulsivity (Gillan et al, 2020;Gillan et al, 2016;Patzelt et al, 2019;Seow et al, 2020;Voon et al, 2015) in a large sample of citizen scientists and by demonstrating comparable psychometrics to the traditional. We then used this task to tackle a question posed by recent studies (Castro-Rodrigues et al, 2022;Feher da Silva & Hare, 2018Kool et al, 2016;Otto, Raio, et al, 2013;Patzelt et al, 2019): are associations between compulsivity and model-based planning dependent on key aspects of the task's design? In a series of within and between-participant experiments, we found the association between model-based planning and compulsivity was greater when using a less deterministic transition ratio structure (70:30 compared to 80:20) and that this association also depends on the specific nature of the drifting reward probabilities used.…”

“…Assessing differences in task motivations, Patzelt et al, (2019) found that offering larger reward amounts increased meanlevel model-based planning levels, but this did not affect its association with compulsivity. Castro-Rodrigues (2022), on the other hand, found evidence to suggest that differences between OCD patients and controls may be smaller when detailed instructions are provided, though the sample (N=46 OCD patients) was too small to test this definitively.…”

“…In another study, they found model-based estimates were significantly greater when participants received in-depth instruction and more practice trials compared to the original task (Feher da Silva & Hare, 2020). More recently, researchers assessed the emergence of model-based planning in a task that was initially absent of any instruction whatsoever (Castro-Rodrigues et al, 2022). They found that only a minority of participants adopted a model-based approach to solving a two-step task without instruction, and once instructions were provided, model-based planning estimates rose rapidly.…”

Human Randomness in the Rock-Paper-Scissors Game

“…This is evident in the fact that people have used objects and natural phenomena to apply randomness in gambling since ancient times [42]. In terms of search methods, for example, some results demonstrate that methods that consider chaos, which is a deterministic phenomenon, give more efficient search results than the method with randomness [43,44]. Certain features of near randomness but not belonging to randomness allow for the creation of special functions, such as efficient search.…”