Modular meta-learning

Alet, Ferran; Lozano-Pérez, Tomás; Kaelbling, Leslie Pack

doi:10.48550/arxiv.1806.10166

Cited by 19 publications

(27 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Expansion-based methods (Rusu et al, 2016;Fernando et al, 2017;Aljundi et al, 2017;Rosenbaum et al, 2018;Chang et al, 2018;Xu and Zhu, 2018;Ferran Alet, 2018;Veniat et al, 2021) add new network modules or experts for new tasks. By construction, expansion-based methods can have zeroforgetting, but their memory complexity can increase super-linearly with the number of tasks, making them undesirable when the pool of tasks is large.…”

Section: Related Workmentioning

confidence: 99%

Wide Neural Networks Forget Less Catastrophically

Mirzadeh¹,

Chaudhry²,

Yin³

et al. 2021

Preprint

View full text Add to dashboard Cite

A growing body of research in continual learning is devoted to overcoming the "Catastrophic Forgetting" of neural networks by designing new algorithms that are more robust to the distribution shifts. While the recent progress in continual learning literature is encouraging, our understanding of what properties of neural networks contribute to catastrophic forgetting is still limited. To address this, instead of focusing on continual learning algorithms, in this work, we focus on the model itself and study the impact of "width" of the neural network architecture on catastrophic forgetting, and show that width has a surprisingly significant effect on forgetting. To explain this effect, we study the learning dynamics of the network from various perspectives such as gradient norm and sparsity, orthogonalization, and lazy training regime. We provide potential explanations that are consistent with the empirical results across different architectures and continual learning benchmarks.

show abstract

Section: Related Workmentioning

confidence: 99%

Wide Neural Networks Forget Less Catastrophically

Mirzadeh¹,

Chaudhry²,

Yin³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The hypothesis that modularity could improve flexibility of learning systems has motivated much empirical work in designing factorized architectures (Devin et al, 2017;Andreas et al, 2016;Chang et al, 2018;Goyal et al, 2019;Kirsch et al, 2018;Alet et al, 2018;Pathak et al, 2019) and reinforcement learners (Simpkins & Isbell, 2019;Sprague & Ballard, 2003;Samejima et al, 2003), but the extent to which the heuristics used in these methods enforce the learnable components to be independently modifiable has yet to be tested. Conversely, other works begin by defining a multiagent system of independently modifiable components and seek methods to induce their cooperation with respect to a global objective (Balduzzi, 2014;Baum, 1996;Srivastava et al, 2013;Chang et al, 2020;Gemp et al, 2020;Balduzzi et al, 2020), but the precise property of a learning system that characterizes its modularity has not been discussed in these works, as far as we are aware.…”

Section: Related Workmentioning

confidence: 99%

Modularity in Reinforcement Learning via Algorithmic Independence in Credit Assignment

Chang,

Kaushik,

Levine

et al. 2021

Preprint

View full text Add to dashboard Cite

Many transfer problems require re-using previously optimal decisions for solving new tasks, which suggests the need for learning algorithms that can modify the mechanisms for choosing certain actions independently of those for choosing others. However, there is currently no formalism nor theory for how to achieve this kind of modular credit assignment. To answer this question, we define modular credit assignment as a constraint on minimizing the algorithmic mutual information among feedback signals for different decisions. We introduce what we call the modularity criterion for testing whether a learning algorithm satisfies this constraint by performing causal analysis on the algorithm itself. We generalize the recently proposed societal decision-making framework as a more granular formalism than the Markov decision process to prove that for decision sequences that do not contain cycles, certain single-step temporal difference action-value methods meet this criterion while all policy-gradient methods do not. Empirical evidence suggests that such action-value methods are more sample efficient than policy-gradient methods on transfer problems that require only sparse changes to a sequence of previously optimal decisions.It is causality that gives us this modularity, and when we lose causality, we lose modularity. Judea Pearl (Ford, 2018) ICML (long) oral presentation: https://sites. google.com/view/modularcreditassignment.

show abstract

“…Parascandolo et al (2017) propose an algorithm to recover a set of independent causal mechanisms by establishing competition between mechanisms, hence driving specialization. Alet et al (2018) proposed a meta learing algorithm to recover a set of specialized modules, but did not establish any connections to causal mechanisms. More recently, Dasgupta et al (2019) adopted a meta-learning approach to draw causal inferences from purely observational data.…”

Section: Related Workmentioning

confidence: 99%

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Bengio¹,

Tristan²,

Rahaman³

et al. 2019

Preprint

View full text Add to dashboard Cite

We propose to meta-learn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional changes, e.g. due to interventions, actions of agents and other sources of non-stationarities. We show that under this assumption, the correct causal structural choices lead to faster adaptation to modified distributions because the changes are concentrated in one or just a few mechanisms when the learned knowledge is modularized appropriately. This leads to sparse expected gradients and a lower effective number of degrees of freedom needing to be relearned while adapting to the change. It motivates using the speed of adaptation to a modified distribution as a meta-learning objective. We demonstrate how this can be used to determine the cause-effect relationship between two observed variables. The distributional changes do not need to correspond to standard interventions (clamping a variable), and the learner has no direct knowledge of these interventions. We show that causal structures can be parameterized via continuous variables and learned end-to-end. We then explore how these ideas could be used to also learn an encoder that would map low-level observed variables to unobserved causal variables leading to faster adaptation out-of-distribution, learning a representation space where one can satisfy the assumptions of independent mechanisms and of small and sparse changes in these mechanisms due to actions and non-stationarities.

show abstract

Modular meta-learning

Cited by 19 publications

References 22 publications

Wide Neural Networks Forget Less Catastrophically

Wide Neural Networks Forget Less Catastrophically

Modularity in Reinforcement Learning via Algorithmic Independence in Credit Assignment

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Contact Info

Product

Resources

About