Automated Curriculum Learning for Neural Networks

Graves, Alex; Bellemare, Marc G.; Menick, Jacob; Munos, Rémi; Kavukcuoglu, Koray

doi:10.48550/arxiv.1704.03003

Cited by 43 publications

(68 citation statements)

References 18 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work we proposed an approach that relies on in the difficulty level of the environmental conditions. An alternative criterion, that has been explored in the context of supervised learning (see Graves et al, 2017;Matiisen et al, 2019) and reinforcement learning (see Portelas et al, 2020) is learning progress, namely the propensity of examples or of environmental conditions to induce learning progress.…”

Section: Discussionmentioning

confidence: 99%

Automated curriculum learning for embodied agents a neuroevolutionary approach

Milano

Nolfi

2021

Sci Rep

View full text Add to dashboard Cite

We demonstrate how the evolutionary training of embodied agents can be extended with a curriculum learning algorithm that automatically selects the environmental conditions in which the evolving agents are evaluated. The environmental conditions are selected to adjust the level of difficulty to the ability level of the current evolving agents, and to challenge the weaknesses of the evolving agents. The method does not require domain knowledge and does not introduce additional hyperparameters. The results collected on two benchmark problems, that require to solve a task in significantly varying environmental conditions, demonstrate that the method proposed outperforms conventional learning methods and generates solutions which are robust to variations and able to cope with different environmental conditions.

show abstract

Section: Discussionmentioning

confidence: 99%

Automated curriculum learning for embodied agents a neuroevolutionary approach

Milano

Nolfi

2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Student-teacher algorithms have been used in multiple works, with the general premise that one algorithm (teacher) is trained to train another (student) (Fan et al, 2018;Liu et al, 2017a). These methods have also been used with a curriculum (Bengio et al, 2009), where the curriculum is either pre-defined and exploited by the teacher (El-Bouri et al, 2020) or implicitly learned by the teacher during training (Graves et al, 2017). Federated learning is a method of training a model (in our case a deep neural network) by using data from multiple centres, without having central access to each of them (McMahan et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

Towards Scheduling Federated Deep Learning using Meta-Gradients for Inter-Hospital Learning

el-Bouri,

Zhu,

Clifton

2021

Preprint

View full text Add to dashboard Cite

Given the abundance and ease of access of personal data today, individual privacy has become of paramount importance, particularly in the healthcare domain. In this work, we aim to utilise patient data extracted from multiple hospital data centres to train a machine learning model without sacrificing patient privacy. We develop a scheduling algorithm in conjunction with a studentteacher algorithm that is deployed in a federated manner. This allows a central model to learn from batches of data at each federal node. The teacher acts between data centres to update the main task (student) algorithm using the data that is stored in the various data centres. We show that the scheduler, trained using meta-gradients, can effectively organise training and as a result train a machine learning model on a diverse dataset without needing explicit access to the patient data. We achieve state-of-the-art performance and show how our method overcomes some of the problems faced in the federated learning such as node poisoning. We further show how the scheduler can be used as a mechanism for transfer learning, allowing different teachers to work together in training a student for state-of-the-art performance.

show abstract

“…Silver et al (2017) use self-play with deep reinforcement learning techniques to master the game of Go; and self-play has even been applied in Dota 5v5 (OpenAI, 2018). Curriculum learning is widely used for training neural networks (see e.g., Bengio et al, 2009;Graves et al, 2017;Elman, 1993;Olsson, 1995). A general framework for automatic task selection is Powerplay (Schmidhuber, 2013;Srivastava et al, 2013), which proposes an asymptotically optimal way of selecting tasks from a set of tasks with program search, and use replay buffer to avoid catastrophic forgetting.…”

Section: Related Workmentioning

confidence: 99%

Competitive Experience Replay

Liu,

Trott,

Socher

et al. 2019

Preprint

View full text Add to dashboard Cite

Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of RL in the real world since both reinforcement learning and domain-specific knowledge are required. It is therefore of great practical importance to develop algorithms which can learn from a binary signal indicating successful task completion or other unshaped, sparse reward signals. We propose a novel method called competitive experience replay, which efficiently supplements a sparse reward by placing learning in the context of an exploration competition between a pair of agents. Our method complements the recently proposed hindsight experience replay (HER) by inducing an automatic exploratory curriculum. We evaluate our approach on the tasks of reaching various goal locations in an ant maze and manipulating objects with a robotic arm. Each task provides only binary rewards indicating whether or not the goal is achieved. Our method asymmetrically augments these sparse rewards for a pair of agents each learning the same task, creating a competitive game designed to drive exploration. Extensive experiments demonstrate that this method leads to faster converge and improved task performance.

show abstract

Automated Curriculum Learning for Neural Networks

Cited by 43 publications

References 18 publications

Automated curriculum learning for embodied agents a neuroevolutionary approach

Automated curriculum learning for embodied agents a neuroevolutionary approach

Towards Scheduling Federated Deep Learning using Meta-Gradients for Inter-Hospital Learning

Competitive Experience Replay

Contact Info

Product

Resources

About