Learning with Feature-Dependent Label Noise: A Progressive Approach

Zhang, Yikai; Zheng, Songzhu; Wu, Pengxiang; Goswami, Mayank; Chen, Chao

doi:10.48550/arxiv.2103.07756

Cited by 8 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Robust learning algorithms include robust networks [2,14,15], robust loss functions [16][17][18][19], and robust regularization [8,32]. Noise detection algorithms include sample selection [20][21][22][23] and pseudolabeling [33][34][35][36][37]. ELR+ [24], DivideMix [25], and LongReMix [38] which combine several LNL and SSL techniques, are state-of-the-art methods in the field of LNL.…”

Section: Learning With Noisy Labelsmentioning

confidence: 99%

CRAS: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels

Higashimoto,

Yoshida,

Muneyasu

2024

Applied Sciences

View full text Add to dashboard Cite

This paper addresses the performance degradation of deep neural networks caused by learning with noisy labels. Recent research on this topic has exploited the memorization effect: networks fit data with clean labels during the early stages of learning and eventually memorize data with noisy labels. This property allows for the separation of clean and noisy samples from a loss distribution. In recent years, semi-supervised learning, which divides training data into a set of labeled clean samples and a set of unlabeled noisy samples, has achieved impressive results. However, this strategy has two significant problems: (1) the accuracy of dividing the data into clean and noisy samples depends strongly on the network’s performance, and (2) if the divided data are biased towards the unlabeled samples, there are few labeled samples, causing the network to overfit to the labels and leading to a poor generalization performance. To solve these problems, we propose the curriculum regularization and adaptive semi-supervised learning (CRAS) method. Its key ideas are (1) to train the network with robust regularization techniques as a warm-up before dividing the data, and (2) to control the strength of the regularization using loss weights that adaptively respond to data bias, which varies with each split at each training epoch. We evaluated the performance of CRAS on benchmark image classification datasets, CIFAR-10 and CIFAR-100, and real-world datasets, mini-WebVision and Clothing1M. The findings demonstrate that CRAS excels in handling noisy labels, resulting in a superior generalization and robustness to a range of noise rates, compared with the existing method.

show abstract

Section: Learning With Noisy Labelsmentioning

confidence: 99%

CRAS: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels

Higashimoto,

Yoshida,

Muneyasu

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…In this section, we compared our method to most recent state-of-the-art methods: DivideMix (Li et al, 2020a), LossModelling (Arazo et al, 2019), Coteaching+ (Yu et al, 2019), Mixup (Zhang et al, 2017), F-correction (Patrini et al, 2017), SELFIE (Song et al, 2019), PLC (Zhang et al, 2021), PENCIL (Yi and Wu, 2019), ELR (Liu et al, 2020a), NCT , MOIT+ (Ortego et al, 2021), NGC , RRL (Li et al, 2020b), FaMUS , GJS (Ghosh and Lan, 2021), PDLC (Liu et al, 2020b). We show, that the proposed method achieves consistent improvements in all datasets and at all noise types and ratios.…”

Section: Synthetic Noisy Datasets Evaluationmentioning

confidence: 99%

S3: Supervised Self-supervised Learning under Label Noise

Feng

Tzimiropoulos

Patras

2021

Preprint

View full text Add to dashboard Cite

Despite the large progress in supervised learning with Neural Networks, there are significant challenges in obtaining high-quality, large-scale and accurately labeled datasets. In this context, in this paper we address the problem of classification in the presence of label noise and more specifically, both close-set and open-set label noise, that is when the true label of a sample may, or may not belong to the set of the given labels. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space; a relabeling mechanism that relies on the confidence of the classifier across subsequent iterations; and a training strategy that trains the encoder both with a self-consistency loss and the classifier-encoder with the cross-entropy loss on the selected samples alone. Without bells and whistles, such as co-training so as to reduce the self-confirmation bias, and with robustness with respect to settings of its few hyper-parameters, our method significantly surpasses previous methods on both CIFAR10/CIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.

show abstract

“…The experiment results show that the iterative label cleaning strategy achieves the best performance. Following this conclusion, we utilize a state-of-theart cleaning training method [65] that iteratively corrects labels with predicted probabilities above a decreasing threshold.…”

Section: Learning From Noisy Labelsmentioning

confidence: 99%

One-shot Weakly-Supervised Segmentation in Medical Images

Lei¹,

Su²,

Gu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks usually require accurate and a large number of annotations to achieve outstanding performance in medical image segmentation. One-shot segmentation and weakly-supervised learning are promising research directions that lower labeling effort by learning a new class from only one annotated image and utilizing coarse labels instead, respectively. Previous works usually fail to leverage the anatomical structure and suffer from class imbalance and low contrast problems. Hence, we present an innovative framework for 3D medical image segmentation with one-shot and weakly-supervised settings. Firstly a propagation-reconstruction network is proposed to project scribbles from annotated volume to unlabeled 3D images based on the assumption that anatomical patterns in different human bodies are similar. Then a dual-level feature denoising module is designed to refine the scribbles based on anatomical-and pixel-level features. After expanding the scribbles to pseudo masks, we could train a segmentation model for the new class with the noisy label training strategy. Experiments on one abdomen and one headand-neck CT dataset show the proposed method obtains significant improvement over the state-of-the-art methods and performs robustly even under severe class imbalance and low contrast.

show abstract

Learning with Feature-Dependent Label Noise: A Progressive Approach

Cited by 8 publications

References 21 publications

CRAS: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels

CRAS: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels

S3: Supervised Self-supervised Learning under Label Noise

One-shot Weakly-Supervised Segmentation in Medical Images

Contact Info

Product

Resources

About