Rich and lazy learning of task representations in brains and neural networks

Flesch, Timo; Juechems, Keno; Dumbalska, Tsvetomira; Saxe, Andrew M.; Summerfield, Christopher

doi:10.1101/2021.04.23.441128

Cited by 25 publications

(35 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Synaptic weights were initialised from a zero-centered Gaussian distribution with standard deviation 𝜎 = 𝑔 * √1 𝑓𝑎𝑛_𝑖𝑛 ⁄ where 𝑔 = 0.025 and 𝑔 = 1 in hidden and readout layers respectively. The hidden layer weights were initialised to small values to encourage a lowdimensional ("rich") solution [32]. We employed a training procedure very similar to that used for human subjects.…”

Section: Neural Network Simulationsmentioning

confidence: 99%

Neural knowledge assembly in humans and deep networks

Nelli

Braun

Dumbalska

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Human understanding of the world can change rapidly when new information comes to light, such as when a plot twist occurs in a work of fiction. This flexible knowledge assembly requires few-shot reorganisation of neural codes for relations among objects and events. However, existing computational theories are largely silent about how this could occur. Here, participants learned a transitive ordering among novel objects within two distinct contexts, before exposure to new knowledge revealing how the contexts were linked. BOLD signals in dorsal frontoparietal cortical areas revealed that objects were rapidly and dramatically rearranged on the neural manifold after minimal exposure to the linking information. We then adapt stochastic online gradient descent to permit similar rapid knowledge assembly in a neural network model.

show abstract

Section: Neural Network Simulationsmentioning

confidence: 99%

Neural knowledge assembly in humans and deep networks

Nelli

Braun

Dumbalska

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The representations learned by an ANN can depend strongly on the training regime. Prior research has shown that small alterations to weight initialization parameters can greatly impact the structure of the learned hidden representations in ANNs 21,22 . Specifically, those studies found during a "rich" training regime (in which network initializations had small weight variances), ANNs learned lower-dimensional and structured representations.…”

Section: Compression-then-expansion Of Task Representations In a Feedforward Ann Emerges During Rich Trainingmentioning

confidence: 99%

“…Next, we characterized properties of the learned ANN weights. In line with previous work, we first calculated the Frobenius norm of weights under different weight initializations 21 .…”

Section: Richly Trained Anns Learn Hierarchical Representational Transformationsmentioning

confidence: 99%

Multi-task representations in human cortex transform along a sensory-to-motor hierarchy

Ito

Murray

2021

Preprint

View full text Add to dashboard Cite

Human cognition recruits diverse neural processes, yet the organizing computational and functional architectures remain unclear. Here, we characterized the geometry and topography of multi-task representations across human cortex using functional MRI during 26 cognitive tasks in the same subjects. We measured the representational similarity across tasks within a region, and the alignment of representations between regions. We found a cortical topography of representational alignment following a hierarchical sensory-association-motor gradient, revealing compression-then-expansion of multi-task dimensionality along this gradient. To investigate computational principles of multi-task representations, we trained multi-layer neural network models to transform empirical visual to motor representations. Compression-then-expansion organization in models emerged exclusively in a training regime where internal representations are highly optimized for sensory-to-motor transformation, and not under generic signal propagation. This regime produces hierarchically structured representations similar to empirical cortical patterns. Together, these results reveal computational principles that organize multi-task representations across human cortex to support flexible cognition.

show abstract

“…Thus, the identity and number of highvariance dimensions will likely vary across tasks, as will the basic response features captured by those dimensions. This possibility is supported by network models that perform multiple tasks (Duncker et al, 2020;Flesch et al, 2021;Logiaco et al, 2019) or subtasks (Zimnik and Churchland, 2021). When different tasks require very different dynamics, a very natural way to 'switch' dynamics is to alter the occupied subspace.…”

Section: Task-specific Subspacesmentioning

confidence: 99%

Cortical control of virtual self-motion using task-specific subspaces

Schroeder

Perkins

Wang

et al. 2019

Preprint

View full text Add to dashboard Cite

1Brain-machine interfaces (BMIs) for reaching have enjoyed continued performance 2improvements. Yet there remains significant need for locomotor BMIs (e.g., for wheelchair 3 control), which could potentially benefit a much larger patient population. Fewer studies have 4 addressed this need, and the most effective approach remains undetermined. Here, we develop 5 a locomotor BMI based on cortical activity as monkeys cycle a hand-held pedal to progress 6 along a virtual track. Unlike most reach-based BMIs, we did not directly map neural states to 7 commanded velocity or position. Instead, we leveraged features of the neural population 8 response that were robust during rhythmic cycling. These included an overall shift in neural 9 state when moving, and rotational trajectories with direction-specific paths. We used nonlinear 10 means to infer kinematics from these features. Online BMI-control success rates approached 11 those during manual control. Our results illustrate that different use-cases can require very 12 different approaches to guiding a prosthetic via neural activity. 13 14 Brain-machine interfaces (BMIs) interpret neural activity and provide control signals to external 15 devices such as computers and prosthetic limbs. Intracortical BMIs for reach-like tasks have 16 proved successful in primates and human clinical trials 1-8 . More widespread use appears 17imminent. Yet at the same time, there exist non-reach-like movements whose restoration is 18 valuable to patients. For example, many patients could benefit from a BMI that controls 19 locomotion through their environment (e.g., movement of a wheelchair). Recent work has 20 demonstrated that this is feasible 9,10 . While locomotor BMIs can be guided by reach-inspired 21 decoding approaches, other viable strategies exist and remain unexplored. For example, it may 22A decoder that leveraged these dominant features provided excellent online control of virtual 71 locomotion. Success rates and acquisition times were very close to those achieved under manual 72 control. Almost no training or adaptation time was needed; the low-latency and accuracy of the 73 decoder were such that monkeys appeared to barely notice transitions from manual control to 74 BMI control. These results demonstrate the feasibility of BMI locomotion based on rhythmic 75 neural activity. More broadly, they establish that opportunistic decode strategies can work well 76 in non-reach-based scenarios, but that new applications require novel decode approaches that 77 respect the dominant structure of neural activity. 78 79 80 5 Results 81Behavior 82We trained two monkeys (G and E) to rotate a hand-held pedal to move through a virtual 83 environment (Fig. 1). All motion was along a linear track -no steering was necessary. 84Consistent with this, a single pedal was cycled with the right arm only. Our goal when 85 decoding was to reconstruct the virtual motion produced by that single pedal. On each trial, a 86 target appeared in the distance. To acquire that target, monkeys produced virtual ve...

show abstract

Rich and lazy learning of task representations in brains and neural networks

Cited by 25 publications

References 44 publications

Neural knowledge assembly in humans and deep networks

Neural knowledge assembly in humans and deep networks

Multi-task representations in human cortex transform along a sensory-to-motor hierarchy

Cortical control of virtual self-motion using task-specific subspaces

Contact Info

Product

Resources

About