Training Classifiers that are Universally Robust to All Label Noise Levels

Abstract

For classification tasks, deep neural networks are prone to overfitting in the presence of label noise. Although existing methods are able to alleviate this problem at low noise levels, they encounter significant performance reduction at high noise levels, or even at medium noise levels when the label noise is asymmetric. To train classifiers that are universally robust to all noise levels, and that are not sensitive to any variation in the noise model, we propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. In particular, we shall assume that a small subset of any given noisy dataset is known to have correct labels, which we treat as "positive", while the remaining noisy subset is treated as "unlabeled". Our framework consists of the following two components: (1) We shall generate, via iterative updates, an augmented clean subset with additional reliable "positive" samples filtered from "unlabeled" samples; (2) We shall train a teacher model on this larger augmented clean set. With the guidance of the teacher model, we then train a student model on the whole dataset. Experiments were conducted on the CIFAR-10 dataset with synthetic label noise at multiple noise levels for both symmetric and asymmetric noise. The results show that our framework generally outperforms at medium to high noise levels. We also evaluated our framework on Clothing1M, a real-world noisy dataset, and we achieved 2.94% improvement in accuracy over existing state-of-the-art methods.

GenKL: An Iterative Framework for Resolving Label Ambiguity and Label Non-conformity in Web Images Via a New Generalized KL Divergence

“…• We propose GenKL, a training framework that is robust to NC instances in training data. • In our experiments on web image datasets, Clothing1M (Xiao et al, 2015), Food101/ Food101N Lee et al, 2018) and mini WebVision 1.0 (Li et al, 2017), we achieved new SOTA accuracies 81.34%, 85.73%, and 78.99%/92.54% (top-1/top-5), respectively.…”

“…Web data is an abundant source for curating image datasets Kaur et al, 2017;Lee et al, 2018;Liang et al, 2020;Shang et al, 2018;Xiao et al, 2015). Raw web images collected online are typically annotated with weak-supervision methods (Xiao et al 2015;Varma & Ré 2018;Tekumalla & Banda 2021;Helmstetter & Paulheim 2021;Yang et al 2022).…”

“…Web data is an abundant source for curating image datasets Kaur et al, 2017;Lee et al, 2018;Liang et al, 2020;Shang et al, 2018;Xiao et al, 2015). Raw web images collected online are typically annotated with weak-supervision methods (Xiao et al 2015;Varma & Ré 2018;Tekumalla & Banda 2021;Helmstetter & Paulheim 2021;Yang et al 2022). Although much more efficient in comparison to manual annotation, such automated annotation methods inevitably introduce nonconforming (NC) instances, which comprise both ambiguous in-distribution (ID) instances and out-of-distribution (OOD) B Kai Fong Ernest Chong ernest_chong@sutd.edu.sg Xia Huang xia_huang@mymail.sutd.edu.sg 1 Singapore University of Technology and Design, 8 Somapah Road, Singapore 487372, Singapore instances.…”

“…Although much more efficient in comparison to manual annotation, such automated annotation methods inevitably introduce nonconforming (NC) instances, which comprise both ambiguous in-distribution (ID) instances and out-of-distribution (OOD) B Kai Fong Ernest Chong ernest_chong@sutd.edu.sg Xia Huang xia_huang@mymail.sutd.edu.sg 1 Singapore University of Technology and Design, 8 Somapah Road, Singapore 487372, Singapore instances. For example, Clothing1M (Xiao et al, 2015), a large-scale web image dataset that is well-known for containing real-world label noise, also contains NC instances; see Fig. 1 for explicit examples.…”

FedCorr: Multi-Stage Federated Learning for Label Noise Correction

“…Label correction and, more generally, methods to deal with label noise, are well-studied in CL. Yet, even state-ofthe-art CL methods for tackling label noise [3,8,9,18,30,33,35,40], when applied to local clients, are inadequate in mitigating the performance degradation in the FL setting, due to the limited sizes of local datasets. These CL methods cannot be applied on the global sever or across multiple clients due to FL privacy requirements.…”