Safeguarded Dynamic Label Regression for Noisy Supervision

Abstract

Learning with noisy labels is imperative in the Big Data era since it reduces expensive labor on accurate annotations. Previous method, learning with noise transition, has enjoyed theoretical guarantees when it is applied to the scenario with the class-conditional noise. However, this approach critically depends on an accurate pre-estimated noise transition, which is usually impractical. Subsequent improvement adapts the preestimation in the form of a Softmax layer along with the training progress. However, the parameters in the Softmax layer are highly tweaked for the fragile performance and easily get stuck into undesired local minimums. To overcome this issue, we propose a Latent Class-Conditional Noise model (LCCN) that models the noise transition in a Bayesian form. By projecting the noise transition into a Dirichlet-distributed space, the learning is constrained on a simplex instead of some adhoc parametric space. Furthermore, we specially deduce a dynamic label regression method for LCCN to iteratively infer the latent true labels and jointly train the classifier and model the noise. Our approach theoretically safeguards the bounded update of the noise transition, which avoids arbitrarily tuning via a batch of samples. Extensive experiments have been conducted on controllable noise data with CIFAR10 and CIFAR-100 datasets, and the agnostic noise data with Clothing1M and WebVision17 datasets. Experimental results have demonstrated that the proposed model outperforms several state-of-the-art methods.

Towards Robust Learning with Different Label Noise Distributions

“…Loss correction approaches [16], [18], [24], [25] either modify the loss directly or the network probabilities to compensate for the incorrect guidance provided by the noisy samples. [25] extend the loss with a perceptual term that introduces a reliance on the model prediction.…”

“…Patrini et al [24] estimate the label noise transition matrix T , which specifies the probability of one label being flipped to another, and correct the softmax probability by multiplying by T . In the same spirit, Yao et al [18] propose to estimate T in a Bayesian non-parametric form and deduce a dynamic label regression method to train the classifier and model the noise.…”

“…Several alternatives exist: semi-supervised learning (SSL), which jointly learns from few labeled images and extensive unlabeled ones [11], [12]; self-supervised learning, where data provides the supervision [9], [13]; or learning with label noise, where automatic labeling processes introduce noise in the observed labels [14], [15]. This paper focuses on this last alternative, which has attracted much recent interest [15]- [18].…”

“…correctly labeled images [19], [20], thus exhibiting lower loss than images with noisy or incorrect labels. This phenomenon is commonly named small loss [14], [15], [18] and recent works exploit this small loss trick to identify clean and noisy samples [16], [21], [22].…”

Disjoint Contrastive Regression Learning for Multi-Sourced Annotations

“…Some approaches [1,2,3,4,5,6] deal with the noisylabel issue or labeling biases from crowd-sourced annotations. Many advances [7,8,9,10] have been made in recent years, especially for classification tasks with crowd-sourced labels. For regression tasks, except for the inconsistency among multiple annotators, the intra-annotator consistency also needs to be considered, in which the labeled rankings of samples from the same annotator usually align well with the rankings of the latent ground truth scores.…”