Frontostriatal anatomical connections predict age- and difficulty-related differences in reinforcement learning

Abstract

Reinforcement learning (RL) is supported by a network of striatal and frontal cortical structures that are connected through white-matter fiber bundles. With age, the integrity of these white-matter connections declines. The role of structural frontostriatal connectivity in individual and age-related differences in RL is unclear, although local white-matter density and diffusivity have been linked to individual differences in RL. Here we show that frontostriatal tract counts in young human adults (aged 18-28), as assessed noninvasively with diffusion-weighted magnetic resonance imaging and probabilistic tractography, positively predicted individual differences in RL when learning was difficult (70% valid feedback). In older adults (aged 63-87), in contrast, learning under both easy (90% valid feedback) and difficult conditions was predicted by tract counts in the same frontostriatal network. Furthermore, network-level analyses showed a double dissociation between the task-relevant networks in young and older adults, suggesting that older adults relied on different frontostriatal networks than young adults to obtain the same task performance. These results highlight the importance of successful information integration across striatal and frontal regions during RL, especially with variable outcomes.

Relevance of working memory for reinforcement learning in older adults varies with timescale of learning

“…Indeed, age-related changes in dopaminergic processing are not limited to the striatum: a recent meta-analysis suggested similar age-related declines in dopaminergic receptor and transporter availability in the striatum, frontal cortex, and midbrain (Karrer et al, 2017). Recently, we have also demonstrated that different networks of frontostriatal connections are related to RL performance in older compared to young adults, including additional tracts between the striatum and lateral PFC (van de Vijver et al, 2016). Thus, further research including neuroimaging measures is needed to increase our understanding of age-related changes in the balance between brain networks related to WM and low-level, RPE-based learning to support RL.…”

“…Additionally, it is a first indication of how the mechanisms supporting RL may vary depending on the timescale of learning. As older adults seem to use different frontostriatal brain networks than young adults do to process performance feedback and improve behavior (van de Vijver et al, 2016;van de Vijver, Cohen, & Ridderinkhof, 2014), the investigation of the interplay between cortical and subcortical changes and the related cognitive implications may provide promising insights into the dynamics of age-related changes in behavioral adaptation, and the differences therein with different timescales of learning.…”

“…Behavioral updating based on previous experiences declines with age: older adults show increased error rates and may never reach the same level of accuracy as young adults (see, e.g., Eppinger & Kray, 2011;Hämmerer, Li, Müller, & Lindenberger, 2011;Mell et al, 2005;Simon, Howard, & Howard, 2010;van de Vijver, Ridderinkhof, et al, 2016;Weiler, Bellebaum, & Daum, 2008). The extent of this decline depends both on specific characteristics of the aging individual and on task demands (Eppinger, Hämmerer, & Li, 2011;Hämmerer & Eppinger, 2012;Ligneul, 2019;van de Vijver, Ridderinkhof, et al, 2015).…”

Learning in anticipation of reward and punishment: Perspectives across the human lifespan

“…However, learning the correct contingencies of the orthogonalized go/no-go task may critically rely on high-level cognitive functions, thus lifespan differences reported here may also relate to interindividual differences in working memory and long-term memory. These cognitive functions may also be compromised as a result of age-related decline in grey and white matter integrity (Draganski et al , 2011; Samanez-Larkin et al , 2012, Chowdhury et al , 2013 b ; Callaghan et al , 2014; Acosta-Cabronero et al , 2016; Steiger et al , 2016; van de Vijver et al , 2016) which may influence the ability of the instrumental system to learn the task contingencies as indexed by learning rate, reward and punishment sensitivity. Therefore, the effect of decreased dopamine function on the strength of the Pavlovian system may be shadowed by the effects of an age-related decrease in executive functions or instrumental abilities related to structural decline.…”

Corticostriatal White Matter Integrity and Dopamine D1 Receptor Availability Predict Age Differences in Prefrontal Value Signaling during Reward Learning

“…Integrity of frontostriatal pathways as measured by diffusion weighted imaging (DWI) has previously proven the important of good performance in probabilistic reward learning tasks which measure value-based decision-making ability ( Samanez-Larkin et al. 2012 ; van de Vijver et al. 2016 ).…”