A well-established notion in cognitive neuroscience proposes that multiple brain systems contribute to choice behaviour. These include: (1) a model-free system that uses values cached from the outcome history of alternative actions, and (2) a model-based system that considers action outcomes and the transition structure of the environment. The widespread use of this distinction, across a range of applications, renders it important to index their distinct influences with high reliability. Here we consider the two-stage task, widely considered as a gold standard measure for the contribution of model-based and model-free systems to human choice. We tested the internal/temporal stability of measures from this task, including those estimated via an established computational model, as well as an extended model using drift-diffusion. Drift-diffusion modeling suggested that both choice in the first stage, and RTs in the second stage, are directly affected by a model-based/free trade-off parameter. Both parameter recovery and the stability of model-based estimates were poor but improved substantially when both choice and RT were used (compared to choice only), and when more trials (than conventionally used in research practice) were included in our analysis. The findings have implications for interpretation of past and future studies based on the use of the two-stage task, as well as for characterising the contribution of model-based processes to choice behaviour.

Model-free learning enables an agent to make better decisions based on prior experience while representing only minimal knowledge about an environment’s structure. It is generally assumed that model-free state representations are based on outcome-relevant features of the environment. Here, we challenge this assumption by providing evidence that a putative model-free system assigns credit to task representations that are irrelevant to an outcome. We examined data from 769 individuals performing a well-described 2-step reward decision task where stimulus identity but not spatial-motor aspects of the task predicted reward. We show that participants assigned value to spatial-motor representations despite it being outcome irrelevant. Strikingly, spatial-motor value associations affected behavior across all outcome-relevant features and stages of the task, consistent with credit assignment to low-level state-independent task representations. Individual difference analyses suggested that the impact of spatial-motor value formation was attenuated for individuals who showed greater deployment of goal-directed (model-based) strategies. Our findings highlight a need for a reconsideration of how model-free representations are formed and regulated according to the structure of the environment.

No abstract

No abstract

Learning-based therapies, such as cognitive-behavioral therapy, are used worldwide, and their efficacy is endorsed by health and research funding agencies. However, the mechanisms behind both their strengths and their weaknesses are inadequately understood. Here we describe how advances in computational modeling may help formalize and test hypotheses regarding how patients make inferences, which are core postulates of these therapies. Specifically, we highlight the relevance of computations with regard to the development, maintenance, and therapeutic change in psychiatric disorders. A Bayesian approach helps delineate which apparent inferential biases and aberrant beliefs are in fact near-normative, given patients’ current concerns, and which are not. As examples, we formalize three hypotheses. First, high-level dysfunctional beliefs should be treated as beliefs over models of the world. There is a need to test how, and whether, people apply these high-level beliefs to guide the formation of lower level beliefs important for real-life decision making, conditional on their experiences. Second, during the genesis of a disorder, maladaptive beliefs grow because more benign alternative schemas are discounted during belief updating. Third, we propose that when patients learn within therapy but fail to benefit in real life, this can be accounted for by a mechanism that we term overaccommodation, similar to that used to explain fear reinstatement. Beyond these specifics, an ambitious collaborative research program between computational psychiatry researchers, therapists, and experts-by-experience needs to form testable predictions out of factors claimed to be important for therapy.

The rate of exceptionally slow reaction times (RTs), described by the long tail of the RT distribution, was found to be amplified in a variety of special populations with cognitive deficits (e.g., early-stage Alzheimer's disease, attention-deficit/hyperactivity disorder, low intelligence, elderly). Previous individual differences studies found high correlations between working memory (WM) and parameters that characterize the magnitude of the long-RT tail. However, the causal direction remains unknown. In 3 choice-reaction task experiments, we examined this relationship by directly manipulating WM availability. In Experiment 1, the stimulus-response rules were either arbitrary (WM demanding) or nonarbitrary. In Experiment 2, the arbitrary rules were either novel (demanding) or practiced. In Experiment 3, WM was loaded with either declarative (stimulus-stimulus) or procedural (stimulus-response) arbitrary rules. Using an ex-Gaussian model fitting, we found across all experiments that WM demands uniquely influenced the τ parameter, mostly responsible for the long-RT distribution tail. Evidence accumulation modeling of the choice process indicated that WM load had little influence on the decision process itself and primarily affected the duration of an exponentially distributed nondecision component, assumed to reflect the process of rule retrieval. Theoretical interpretations and implications are discussed.

No abstract

In this study, we tested the proposal that the Stroop task involves two conflicts--task conflict and informational conflict. Task conflict was defined as the latency difference between color words and non-letter neutrals, and manipulated by varying the proportion of color words versus non-letter neutrals. Informational conflict was defined as the latency difference between incongruent and congruent trials and manipulated by varying the congruent-to-incongruent trial ratio. We replicated previous findings showing that increasing the ratio of incongruent-to-congruent trials reduces the latency difference between the incongruent and congruent condition (i.e., informational conflict), as does increasing the proportion of color words (i.e., task conflict). A significant under-additive interaction between the two proportion manipulations (congruent vs. incongruent and color words vs. neutrals) indicated that the effects of task conflict and informational conflict were not additive. By assessing task conflict as the contrast between color words and neutrals, we found that task conflict existed in all of our experimental conditions. Under specific conditions, when task conflict dominated behavior by explaining most of the variability between congruency conditions, we also found negative facilitation, thus demonstrating that this effect is a special case of task conflict.