Meta Learning via Learned Loss

Bechtle, Sarah; Molchanov, Artem; Chebotar, Yevgen; Grefenstette, Edward; Righetti, Ludovic; Sukhatme, Gaurav S.; Meier, Franziska

doi:10.1109/icpr48806.2021.9412010

Cited by 28 publications

(38 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gradient based methods are closely related to meta-gradient methods, as they use second-order derivatives to optimize ζ. Bechtle et al (2020) described a general framework for meta learning with learned loss functions. Most meta-gradient RL algorithms, as mentioned in Section 4.5, start with a human designed loss function L(•) and modify it with parameterization L(θ; ζ) to allow inner loop and outer loop procedures.…”

Section: Learning Reinforcement Learning Algorithmsmentioning

confidence: 99%

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

Parker-Holder,

Rajan,

Song

et al. 2022

Preprint

View full text Add to dashboard Cite

The combination of Reinforcement Learning (RL) with deep learning has led to a series of impressive feats, with many believing (deep) RL provides a path towards generally capable agents. However, the success of RL agents is often highly sensitive to design choices in the training process, which may require tedious and error-prone manual tuning. This makes it challenging to use RL for new problems, while also limits its full potential. In many other areas of machine learning, AutoML has shown it is possible to automate such design choices and has also yielded promising initial results when applied to RL. However, Automated Reinforcement Learning (AutoRL) involves not only standard applications of AutoML but also includes additional challenges unique to RL, that naturally produce a different set of methods. As such, AutoRL has been emerging as an important area of research in RL, providing promise in a variety of applications from RNA design to playing games such as Go. Given the diversity of methods and environments considered in RL, much of the research has been conducted in distinct subfields, ranging from meta-learning to evolution. In this survey we seek to unify the field of AutoRL, we provide a common taxonomy, discuss each area in detail and pose open problems which would be of interest to researchers going forward.

show abstract

Section: Learning Reinforcement Learning Algorithmsmentioning

confidence: 99%

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

Parker-Holder,

Rajan,

Song

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The research of meta learning, also known as "learning to learn", focuses on training models that gain experience and improve performance over multiple learning episodes/tasks [30], which has seen wide adoption in research areas such as reinforcement learning [31,32]. This can be applied in various aspects of learning problems under a bi-level optimization framework [33], including the data set generation [34], learning objective [31,35,36], model architecture [37], initialization parameters [38], and the learning rules [32, 39? ]. Our work studies the meta learning in the context of large scale segment prediction tasks, and further investigates meta learning's application to single task learning with novel auxiliary task family construction.…”

Section: Meta Learningmentioning

confidence: 99%

“…Consider a distribution over tasks p : T → [0, 1], we assume a set of 𝑀 source training (i.e. the support in meta learning literature [36]) and validation (query in meta learning literature [36]) data-sets available sampled from T,…”

Section: Problem Overviewmentioning

confidence: 99%

MetaCon: Unified Predictive Segments System with Trillion Concept Meta-Learning

Li¹,

Hu²,

Palanisamy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate understanding of users in terms of predicative segments play an essential role in the day to day operation of modern internet enterprises. Nevertheless, there are significant challenges that limit the quality of data, especially on long tail predictive tasks. In this work, we present MetaCon, our unified predicative segments system with scalable, trillion concepts meta learning that addresses these challenges. It builds on top of a flat concept representation [1] that summarizes entities' heterogeneous digital footprint, jointly considers the entire spectrum of predicative tasks as a single learning task, and leverages principled meta learning approach with efficient first order meta-optimization procedure under a provable performance guarantee in order to solve the learning task. Experiments on both proprietary production datasets and public structured learning tasks demonstrate that MetaCon can lead to substantial improvements over state of the art recommendation and ranking approaches.

show abstract

“…Metalearning, aka learning to learn, and AutoML have been applied for a wide variety of purposes as summarised in [17,21]. Of particular relevance is meta-learning of loss functions, which has been studied for various purposes including providing differentiable surrogates of nondifferentiable objectives [19], optimising efficiency and asymptotic performance of learning [22,4,18,48,11,12], and improving robustness to train/test domain-shift [3,30]. We are interested in learning white-box losses -i.e., those that can be expressed a short human-readable parametric equation -for efficiency and improved task-transferability compared to neural network alternatives [4,18,3,30], which tend to be less interpretable and need to be learned taskspecifically.…”

Section: Meta-learning Automl and Loss Learningmentioning

confidence: 99%

“…Of particular relevance is meta-learning of loss functions, which has been studied for various purposes including providing differentiable surrogates of nondifferentiable objectives [19], optimising efficiency and asymptotic performance of learning [22,4,18,48,11,12], and improving robustness to train/test domain-shift [3,30]. We are interested in learning white-box losses -i.e., those that can be expressed a short human-readable parametric equation -for efficiency and improved task-transferability compared to neural network alternatives [4,18,3,30], which tend to be less interpretable and need to be learned taskspecifically. Meta-learning of white-box model components has been demonstrated for optimisers [47], activation functions [35], neural architectures [43] and losses for accelerating conventional supervised learning [11,12].…”

Section: Meta-learning Automl and Loss Learningmentioning

confidence: 99%

Searching for Robustness: Loss Learning for Noisy Classification Tasks

Gao

Gouk

Hospedales

2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We present a "learning to learn" approach for discovering white-box classification loss functions that are robust to label noise in the training data. We parameterise a flexible family of loss functions using Taylor polynomials, and apply evolutionary strategies to search for noise-robust losses in this space. To learn re-usable loss functions that can apply to new tasks, our fitness function scores their performance in aggregate across a range of training datasets and architectures. The resulting white-box loss provides a simple and fast "plug-and-play" module that enables effective labelnoise-robust learning in diverse downstream tasks, without requiring a special training procedure or network architecture. The efficacy of our loss is demonstrated on a variety of datasets with both synthetic and real label noise, where we compare favourably to prior work.

show abstract

Meta Learning via Learned Loss

Cited by 28 publications

References 7 publications

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

MetaCon: Unified Predictive Segments System with Trillion Concept Meta-Learning

Searching for Robustness: Loss Learning for Noisy Classification Tasks

Contact Info

Product

Resources

About