Curriculum Learning: A Survey

Soviany, Petru; Ionescu, Radu Tudor; Rota, Paolo; Sebe, Nicu

doi:10.1007/s11263-022-01611-x

Cited by 105 publications

(55 citation statements)

References 143 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Curriculum learning is a method for training machine learning models by gradually increasing the complexity of the task to be solved as well as the data samples used during training in order to improve training efficiency. 18 In the reinforcement learning, curriculum learning is a vital component that ensures the agents receives positive rewards even during early stages of training when the policy is not fully developed. This technique has enabled reinforcement learning agents to learn how to control a quadrupedal robot to walk on challenging terrain 8,19 and complete hiking trails 9 (policies were trained first on flat terrain, then progressively moved to more challenging ones with slopes and obstacles), control realistic bipedal robots in simulation to walk over stepping stones 20 (curriculum is used to generate courses of different complexities), and a 2D simulated obstacle course with different levels.…”

Section: Curriculum Learningmentioning

confidence: 99%

Learning to Guide Multiple Heterogeneous Actors from a Single Human Demonstration via Automatic Curriculum Learning in StarCraft II

Waytowich¹,

Hare²,

Goecks³

et al. 2022

Preprint

View full text Add to dashboard Cite

Traditionally, learning from human demonstrations via direct behavior cloning can lead to high-performance policies given that the algorithm has access to large amounts of high-quality data covering the most likely scenarios to be encountered when the agent is operating. However, in real-world scenarios, expert data is limited and it is desired to train an agent that learns a behavior policy general enough to handle situations that were not demonstrated by the human expert. Another alternative is to learn these policies with no supervision via deep reinforcement learning, however, these algorithms require a large amount of computing time to perform well on complex tasks with high-dimensional state and action spaces, such as those found in StarCraft II. Automatic curriculum learning is a recent mechanism comprised of techniques designed to speed up deep reinforcement learning by adjusting the difficulty of the current task to be solved according to the agent's current capabilities. Designing a proper curriculum, however, can be challenging for sufficiently complex tasks, and thus we leverage human demonstrations as a way to guide agent exploration during training. In this work, we aim to train deep reinforcement learning agents that can command multiple heterogeneous actors where starting positions and overall difficulty of the task are controlled by an automatically-generated curriculum from a single human demonstration. Our results show that an agent trained via automated curriculum learning can outperform stateof-the-art deep reinforcement learning baselines and match the performance of the human expert in a simulated command and control task in StarCraft II modeled over a real military scenario.

show abstract

Section: Curriculum Learningmentioning

confidence: 99%

Learning to Guide Multiple Heterogeneous Actors from a Single Human Demonstration via Automatic Curriculum Learning in StarCraft II

Waytowich¹,

Hare²,

Goecks³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead of improving the annotation quality like automatic re-annotators, noise-tolerant learning intends to minimize the negative impact of the annotation noise during the training period, typically by learning to treat the labels with different degrees of credibility. The most commonly used techniques for mitigating the perturbation of noisy data are soft-labeling (Thiel, 2008) and curriculum learning Soviany et al (2021); Portelas et al (2020); Wang et al (2020); Bengio et al (2009). Though noise-tolerant learning empirically leads to improved robustness for handling annotation noise, our work improves its fundamental component of suspicious annotation detection with an enhanced PLM-based detector and the interpretability of its black-box learning process with annotation correction.…”

Section: Noise-tolerant Learningmentioning

confidence: 99%

“…It helps a human re-annotator start by looking at the most error-prone annotations to save time. The models may benefit from differentiating the reliable data and unreliable data during training in the curriculum learning fashion (Soviany et al, 2021;Wang et al, 2020), or by adjusting the weights of training data (Wang et al, 2019).…”

Section: Definition Of Annotation Inconsistency Detectionmentioning

confidence: 99%

Pre-trained Language Models as Re-Annotators

Chen¹

2022

Preprint

View full text Add to dashboard Cite

Annotation noise are widespread in datasets, but manually revising a flawed corpus is time-consuming and error-prone. Hence, given the prior knowledge in Pre-trained Language Models and the expected uniformity across all annotations, we attempt to reduce annotation noise in the corpus through two tasks automatically: (1) Annotation Inconsistency Detection that indicates the credibility of annotations, and (2) Annotation Error Correction that rectifies the abnormal annotations. We investigate how to acquire semantic sensitive annotation representations from Pre-trained Language Models, expecting to embed the examples with identical annotations to the mutually adjacent positions even without fine-tuning. We proposed a novel credibility score to reveal the likelihood of annotation inconsistencies based on the neighbouring consistency. Then, we fine-tune the Pre-trained Language Models based classifier with cross-validation for annotation correction. The annotation corrector is further elaborated with two approaches: (1) soft labelling by Kernel Density Estimation and (2) a novel distant-peer contrastive loss. We study the re-annotation in relation extraction and create a new manually revised dataset, Re-DocRED, for evaluating document-level re-annotation. The proposed credibility scores show promising agreement with human revisions, achieving a Binary F 1 of 93.4 and 72.5 in detecting inconsistencies on TACRED and DocRED respectively.Moreover, the neighbour-aware classifiers based on distant-peer contrastive learning and uncertain labels achieve Macro F 1 up to 66.2 and 57.8 in correcting annotations on TACRED and DocRED respectively. These improvements are not merely theoretical: Rather, automatically denoised training sets demonstrate up to 3.6% performance improvement for state-of-the-art relation extraction models, and the proposed framework is expected to be hundreds of times faster than the human re-annotators empirically.

show abstract

“…The success of the approach relies in avoiding to force the learning of very difficult examples right from the beginning, instead guiding the model on the right path through the imposed curriculum. This type of curriculum is later referred to as data-level curriculum learning [12]. Indeed, Soviany et al [12] identified several types of curriculum learning approaches in the literature, dividing them into four categories based on the components involved in the definition of machine learning given by Mitchell [13].…”

Section: Introductionmentioning

confidence: 99%

LeRaC: Learning Rate Curriculum

Croitoru¹,

Ristea²,

Ionescu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Most curriculum learning methods require an approach to sort the data samples by difficulty, which is often cumbersome to perform. In this work, we propose a novel curriculum learning approach termed Learning Rate Curriculum (LeRaC), which leverages the use of a different learning rate for each layer of a neural network to create a data-free curriculum during the initial training epochs. More specifically, LeRaC assigns higher learning rates to neural layers closer to the input, gradually decreasing the learning rates as the layers are placed farther away from the input. The learning rates increase at various paces during the first training iterations, until they all reach the same value. From this point on, the neural model is trained as usual. This creates a model-level curriculum learning strategy that does not require sorting the examples by difficulty and is compatible with any neural network, generating higher performance levels regardless of the architecture. We conduct comprehensive experiments on eight datasets from the computer vision (CIFAR-10, CIFAR-100, Tiny ImageNet), language (BoolQ, QNLI, RTE) and audio (ESC-50, CREMA-D) domains, considering various convolutional (ResNet-18, Wide-ResNet-50, DenseNet-121), recurrent (LSTM) and transformer (CvT, BERT, SepTr) architectures, comparing our approach with the conventional training regime. Moreover, we also compare with Curriculum by Smoothing (CBS), a stateof-the-art data-free curriculum learning approach. Unlike CBS, our performance improvements over the standard training regime are consistent across all datasets and models. Furthermore, we significantly surpass CBS in terms of training time (there is no additional cost over the standard training regime for LeRaC).

show abstract

Curriculum Learning: A Survey

Cited by 105 publications

References 143 publications

Learning to Guide Multiple Heterogeneous Actors from a Single Human Demonstration via Automatic Curriculum Learning in StarCraft II

Learning to Guide Multiple Heterogeneous Actors from a Single Human Demonstration via Automatic Curriculum Learning in StarCraft II

Pre-trained Language Models as Re-Annotators

LeRaC: Learning Rate Curriculum

Contact Info

Product

Resources

About