Probing learning through the lens of changes in circuit dynamics

Abstract

Despite the success of dynamical systems as accounts of circuit computation and observed behavior, our understanding of how dynamical systems evolve over learning is very limited. Here we develop a computational framework for extracting core dynamical systems features of recurrent circuits across learning and analyze the properties of these meta-dynamics in model analogues of several brain-relevant tasks. Across learning algorithms and tasks we find a stereotyped path to task mastery, which involves the creation of dynamical systems features and their refinement to a stable solution. This learning universality reveals common principles in the organization of recurrent neural networks in service to function and highlights some of the challenges in reverse engineering learning principles from chronic population recordings of neural activity.

Curriculum learning inspired by behavioral shaping trains neural networks to adopt animal-like decision making strategies

“…Once the wait time task was introduced, features were trimmed from the network's dynamic repertoire stabilizing to the final computational motifs supporting the target task. This expansion and then pruning of dynamical features over training has been previously reported in other tasks [28], and appears to be a common learning trait for attractor-based dynamics. Interestingly, while the STR layer had similar numbers of dynamical features across learning procedures, the total number of dynamical features in the OFC layer at the end of kindergarten CL training conserved a higher number of features than sub-optimal networks trained with classical CL (Fig.…”

“…4d). We calculated the average change in recurrent connection weights over learning (see Methods), and found that large changes in network structure occurred at the beginning of each phase of training for any form of training, as networks reorganize from a random initial state to create low-dimensional manifolds supporting structured behavior [28]. Unexpectedly, we found that kindergarten CL had a dramatic reorganization of the network during the kindergarten phases of training, in particular when learning to perform inference of latent states.…”

“…This choice reflects known functional distinctions between OFC and striatum, but complicates the analysis of dynamics compositionality. For this, it would be useful to trace the evolution of individual dynamics features over learning, to more directly characterize how features developed at one stage of learning support later computations, in the spirit of [28]. Alternatively, it may be useful to consider forms of regularization that encourage functional specialization into subcircuits [39].…”

“…Alternatively, it may be useful to consider forms of regularization that encourage functional specialization into subcircuits [39]. One conserved aspect of learning was that deep RL agents had a point at which they introduced a large number of dynamical systems features in the the OFC layer, and then pruned them later in training, something which appears to be a common learning strategy in attractor-based networks [28]. Detailed analysis of these features showed that these dynamic features were exclusively slow points that acted like line attractors.…”

“…Beyond these qualitative commonalities, the exact geometry of the solution was variable across networks, unlike the “universal” dynamical systems solutions of simpler tasks, where all networks that do well solve the task in the same way and thus are dynamically isomorphic [29, 28]. This task complexity may open interesting avenues for exploring how across-animal behavioral variability can be traced back to variability in their OFC dynamics.…”