A Generalized Framework for Population Based Training

Li, Ang; Spyra, Ola; Perel, Sagi; Dalibard, Valentin; Jaderberg, Max; Gu, Chenjie; Budden, David; Harley, Tim; Gupta, Pramod

doi:10.48550/arxiv.1902.01894

Cited by 1 publication

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To mitigate the sensitivity of experimental results to empirical, and perhaps arbitrary, choices of hyperparameters, we present additional results that leverage Population Based Training (PBT) [26,37] which is a simple asynchronous optimisation algorithm that jointly optimize a population of models and their hyperparameters. In particular, PBT discovers a per-epoch schedule of hyperparameter settings rather than a static fixed configuration used over the entirety of training.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Robust Temporal Ensembling for Learning with Noisy Labels

Brown,

Schifferer,

DiPietro

2021

Preprint

View full text Add to dashboard Cite

Successful training of deep neural networks with noisy labels is an essential capability as most real-world datasets contain some amount of mislabeled data. Left unmitigated, label noise can sharply degrade typical supervised learning approaches. In this paper, we present robust temporal ensembling (RTE), which combines robust loss with semi-supervised regularization methods to achieve noiserobust learning. We demonstrate that RTE achieves state-of-the-art performance across the CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N datasets, while forgoing the recent trend of label filtering and/or fixing. Finally, we show that RTE also retains competitive corruption robustness to unforeseen input noise using CIFAR-10-C, obtaining a mean corruption error (mCE) of 13.50% even in the presence of an 80% noise ratio, versus 26.9% mCE with standard methods on clean data. Robust Temporal Ensembling PreliminariesAdopting the notation of [65], we consider the problem of classification where X ⊂ R d is the feature space and Y = {1, . . . , c} is the label space where the classifier function is a deep neural network with a softmax output layer

show abstract

Section: Methodsmentioning

confidence: 99%

“…We find, as in other studies, that this joint optimization of hyperparameter schedules typically results in faster wall-clock convergence and higher final performance. [25,37] Baselines. To best interpret the effectiveness of RTE, we compare our results to many techniques for learning with noise (Table 1).…”

Section: Methodsmentioning

confidence: 99%