Fast rule switching and slow rule updating in a perceptual categorization task

Abstract

To adapt to a changing world, we must be able to switch between rules already learned and, at other times, learn rules anew. Often we must do both at the same time, switching between known rules while also constantly re-estimating them. Here, we show these two processes, rule switching and rule learning, rely on distinct but intertwined computations, namely fast inference and slower incremental learning. To this end, we studied how monkeys switched between three rules. Each rule was compositional, requiring the animal to discriminate one of two features of a stimulus and then respond with an associated eye movement along one of two different response axes. By modeling behavior we found the animals learned the axis of response using fast inference (rule switching) while continuously re-estimating the stimulus-response associations within an axis (rule learning). Our results shed light on the computational interactions between rule switching and rule learning, and make testable neural predictions for these interactions.

Humans adaptively select different computational strategies in different learning environments.

“…Although flat learning via the RW rule has proven very valuable to describe human (and animal) learning (Glimcher, 2011; Rescorla & Wagner, 1972; Steinberg et al, 2013), a wide range of previous work has argued that flat learning is insufficient to capture human learning in complex environments (Bai et al, 2014; Bouchacourt et al, 2022; Liu et al, 2022; McGuire et al, 2014; Verbeke & Verguts, 2019). Therefore, several hierarchical extensions to the flat learning approach have been proposed in several different environments and data sets (Bai et al, 2014; Behrens et al, 2007; Foucault & Meyniel, 2021; Kruschke, 2008; Mathys et al, 2011; Silvetti et al, 2011; Verbeke et al, 2021).…”

“…This is because high learning rates would lead the agent to “chase the noise” and adapt behavior when the contingencies did not change but low learning rates would significantly decrease the flexibility of the agent in adapting to changing contingencies. As a result, several hierarchical extensions of the RW rule have been proposed to better describe human flexibility (Behrens et al, 2007; Bouchacourt et al, 2022; Mathys et al, 2011; McGuire et al, 2014; Verbeke et al, 2021). Across several reinforcement learning environments, the current article evaluates when flat versus hierarchical learning is computationally beneficial for performance, when it fits human data, and how these two factors relate to one another.…”

“…A second extension consists of storing multiple rule sets and adaptively switch between these sets when there is a large change in stimulus–action reward contingencies (Bouchacourt et al, 2022; Razmi & Nassar, 2022; Verbeke et al, 2021; Verbeke & Verguts, 2019). This extension implements a hierarchical architecture for learning.…”

Building compositional tasks with shared neural subspaces

“…As noted above, monkeys slowly learned whether the S1 or C1 task was in effect 16 . Consistent with this, the decoder performance increased over trials (Fig.…”

Humans adaptively select different computational strategies in different learning environments

“…This is because high learning rates would lead the agent to "chase the noise" and adapt behavior when the contingencies did not change but low learning rates would significantly decrease the flexibility of the agent in adapting to changing contingencies. As a result, several hierarchical extensions of the RW rule have been proposed to better describe human flexibility (Behrens et al, 2007;Bouchacourt et al, 2022;Mathys et al, 2011;McGuire et al, 2014;Verbeke, Ergo, De Loof, & Verguts, 2021).…”