NGC: A Unified Framework for Learning with Open-World Noisy Data

Wu, Zhi-Fan; Wei, Tong; Jiang, Jianwen; Mao, Chaojie; Tang, Mingqian; Li, Yufeng

doi:10.1109/iccv48922.2021.00013

Cited by 45 publications

(41 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Khosla et al (2020) propose supervised contrastive learning (SCL), an approach that aggregates data from the same class as the positive set and obtains improved performance on various supervised learning tasks. The success of SCL has motivated a series of works to apply CL to a number of weakly supervised learning tasks, including noisy label learning (Li et al, 2021a;Wu et al, 2021), semi-supervised learning Zhang et al, 2021), etc. Despite promising empirical results, however, these works, lack theoretical understandings.…”

Section: Related Workmentioning

confidence: 99%

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning

Wang¹,

Xiao²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. In this work, we bridge the gap by addressing two key research challenges in PLL-representation learning and label disambiguation-in one coherent framework. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disambiguation algorithm. PiCO produces closely aligned representations for examples from the same classes and facilitates label disambiguation. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectationmaximization (EM) algorithm perspective. Extensive experiments demonstrate that PiCO significantly outperforms the current state-of-the-art approaches in PLL and even achieves comparable results to fully supervised learning. Code and data available: https://github.com/hbzju/PiCO.

show abstract

Section: Related Workmentioning

confidence: 99%

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning

Wang¹,

Xiao²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Second, the network is fine-tuned on a reliable dataset. Additionally, the methods ProtoMix [31] and NGC [51] work in one stage. They jointly perform the generation of pseudo labels and Sup-CL to combat noisy labels.…”

Section: Contrastive Learningmentioning

confidence: 99%

Selective-Supervised Contrastive Learning with Noisy Labels

Li¹,

Xia²,

Ge³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep networks have strong capacities of embedding data into latent representations and finishing following tasks. However, the capacities largely come from high-quality annotated labels, which are expensive to collect. Noisy labels are more affordable, but result in corrupted representations, leading to poor generalization performance. To learn robust representations and handle noisy labels, we propose selective-supervised contrastive learning (Sel-CL) in this paper. Specifically, Sel-CL extend supervised contrastive learning (Sup-CL), which is powerful in representation learning, but is degraded when there are noisy labels. Sel-CL tackles the direct cause of the problem of Sup-CL. That is, as Sup-CL works in a pair-wise manner, noisy pairs built by noisy labels mislead representation learning. To alleviate the issue, we select confident pairs out of noisy ones for Sup-CL without knowing noise rates. In the selection process, by measuring the agreement between learned representations and given labels, we first identify confident examples that are exploited to build confident pairs. Then, the representation similarity distribution in the built confident pairs is exploited to identify more confident pairs out of noisy pairs. All obtained confident pairs are finally used for Sup-CL to enhance representations. Experiments on multiple noisy datasets demonstrate the robustness of the learned representations by our method, following the state-of-the-art performance. Source codes are available at https://github.com/ShikunLi/Sel-CL

show abstract

“…Another linear layer following the feature extractor is used as classifier. When minimizing L pc , we apply mixup [31] to improve the generalization which has been shown to be effective for learning with noisy labels [29].…”

Section: Example Reweightingmentioning

confidence: 99%

“…long-tailed learning) have been studied for many years. When dealing with label noise, the most popular approach is sample selection where correctly-labeled examples are identified by capturing the training dynamics of DNNs [11,29]. When dealing with class imbalance, many existing works propose to reweight examples or design unbiased loss functions by taking into account the class distribution of training set [26,3,8].…”

Section: Introductionmentioning

confidence: 99%

Prototypical Classifier for Robust Class-Imbalanced Learning

Wei¹,

Shi²,

Li³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks have been shown to be very powerful methods for many supervised learning tasks. However, they can also easily overfit to training set biases, i.e., label noise and class imbalance. While both learning with noisy labels and class-imbalanced learning have received tremendous attention, existing works mainly focus on one of these two training set biases. To fill the gap, we propose Prototypical Classifier, which does not require fitting additional parameters given the embedding network. Unlike conventional classifiers that are biased towards head classes, Prototypical Classifier produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. By leveraging this appealing property, we can easily detect noisy labels by thresholding the confidence scores predicted by Prototypical Classifier, where the threshold is dynamically adjusted through the iteration. A sample reweghting strategy is then applied to mitigate the influence of noisy labels. We test our method on CIFAR-10-LT, CIFAR-100-LT and Webvision datasets, observing that Prototypical Classifier obtains substaintial improvements compared with state of the arts.

show abstract

NGC: A Unified Framework for Learning with Open-World Noisy Data

Cited by 45 publications

References 20 publications

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning

Selective-Supervised Contrastive Learning with Noisy Labels

Prototypical Classifier for Robust Class-Imbalanced Learning

Contact Info

Product

Resources

About