Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels

Wu, Songhua; Xia, Xiaobo; Liu, Tongliang; Han, Bo; Gong, Maoguo; Wang, Nannan; Liu, Haifeng; Niu, Gang

doi:10.48550/arxiv.2006.07831

Cited by 3 publications

(2 citation statements)

References 20 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generally, the algorithms of learning with noisy labels can be divided into two categories: statistically inconsistent algorithms and statistically consistent algorithms. Methods in the rst category are heuristic, such as selecting reliable examples to train model [15,48,50,14,30,35,18], correcting labels [29,19,38,34], and adding regularization [13,12,41,40,26,24,43]. Those methods empirically perform well.…”

Section: Introductionmentioning

confidence: 99%

Do We Need to Penalize Variance of Losses for Learning with Label Noise?

Lin¹,

Yu²,

Du³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels. Intuitively, when there is a nite training sample, penalizing the variance of losses will improve the stability and generalization of the algorithms. Interestingly, we found that the variance should be increased for the problem of learning with noisy labels. Specically, increasing the variance will boost the memorization e ects and reduce the harmfulness of incorrect labels. By exploiting the label noise transition matrix, regularizers can be easily designed to reduce the variance of losses and be plugged in many existing algorithms. Empirically, the proposed method by increasing the variance of losses signi cantly improves the generalization ability of baselines on both synthetic and real-world datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

Do We Need to Penalize Variance of Losses for Learning with Label Noise?

Lin¹,

Yu²,

Du³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Generally, the algorithms for combating noisy labels can be categorized into statistically inconsistent algorithms and statistically consistent algorithms. The statistically inconsistent algorithms are heuristic, such as selecting possible clean examples to train the classifier [9,51,53,11,26,33,14], re-weighting examples to reduce the effect of noisy labels [33,21], correcting labels [25,15,36,32], or adding regularization [10,8,39,38,19,17,41]. These approaches empirically work well, but there is no theoretical guarantee that the learned classifiers can converge to the optimal ones learned from clean data.…”

Section: Introductionmentioning

confidence: 99%

Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network

Yang¹,

Yang²,

Han³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In label-noise learning, estimating the transition matrix is a hot topic as the matrix plays an important role in building statistically consistent classifiers. Traditionally, the transition from clean labels to noisy labels (i.e., clean label transition matrix) has been widely exploited to learn a clean label classifier by employing the noisy data. Motivated by that classifiers mostly output Bayes optimal labels for prediction, in this paper, we study to directly model the transition from Bayes optimal labels to noisy labels (i.e., Bayes label transition matrix) and learn a Bayes optimal label classifier. Note that given only noisy data, it is ill-posed to estimate either the clean label transition matrix or the Bayes label transition matrix. But favorably, Bayes optimal labels are less uncertain compared with the clean labels, i.e., the class posteriors of Bayes optimal labels are one-hot vectors while those of clean labels are not. This enables two advantages to estimate the Bayes label transition matrix, i.e., (a) we could theoretically recover a set of noisy data with Bayes optimal labels under mild conditions; (b) the feasible solution space is much smaller. By exploiting the advantages, we estimate the Bayes label transition matrix by employing a deep neural network in a parameterized way, leading to better generalization and superior classification performance.Preprint. Under review.

show abstract

Instance Correction for Learning with Open-set Noisy Labels

Xia¹,

Liu²,

Han³

et al. 2021

Preprint

View full text Add to dashboard Cite

The problem of open-set noisy labels denotes that part of training data have a di erent label space that does not contain the true class. Lots of approaches, e.g., loss correction and label correction, cannot handle such open-set noisy labels well, since they need training data and test data to share the same label space, which does not hold for learning with open-set noisy labels. The state-of-the-art methods thus employ the sample selection approach to handle open-set noisy labels, which tries to select clean data from noisy data for network parameters updates. The discarded data are seen to be mislabeled and do not participate in training. Such an approach is intuitive and reasonable at rst glance. However, a natural question could be raised "can such data only be discarded during training?". In this paper, we show that the answer is no. Speci cally, we discuss that the instances of discarded data could consist of some meaningful information for generalization. For this reason, we do not abandon such data, but use instance correction to modify the instances of the discarded data, which makes the predictions for the discarded data consistent with given labels. Instance correction are performed by targeted adversarial attacks. The corrected data are then exploited for training to help generalization. In addition to the analytical results, a series of empirical evidences are provided to justify our claims.

show abstract

Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels

Cited by 3 publications

References 20 publications

Do We Need to Penalize Variance of Losses for Learning with Label Noise?

Do We Need to Penalize Variance of Losses for Learning with Label Noise?

Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network

Instance Correction for Learning with Open-set Noisy Labels

Contact Info

Product

Resources

About