Augmentation Strategies for Learning with Noisy Labels

Nishi, Kento; Ding, Yi; Rich, Alex; Höllerer, Tobias

doi:10.1109/cvpr46437.2021.00793

Cited by 68 publications

(54 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In later subsections, we evaluate the sensitivity of the model to Table 1 Comparison with baseline methods and current state-of-the-art approaches on CIFAR-10 and CIFAR-100 with symmetric label noise in test accuracy (%). DivideMix [1] and AugDesc-(SAW/WAW) [21] was reimplemented using public code. The mean accuracy and its standard deviation are computed over five noise realizations.…”

Section: Methodsmentioning

confidence: 99%

“…We compared our method with DivideMix [1] and Augment Descent (AugDesc) [21], which is the current state-of-theart for learning with noisy labels through sample selection and data augmentation, using the same network architecture. AugDesc defines the common random flip and crop image augmentation as weak data augmentation, and AutoAugment [25] as strong data augmentation.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…Most notably, DivideMix [1] uses two networks for sample selection with a two-component mixture model, and applies the semi-supervised learning technique called MixMatch [20]. Recently, AugDesc [21] has been proposed which models the loss distribution with weakly augmented samples for sample selection and trains the model with strongly augmented samples.…”

Section: Learning With Noisy Labelsmentioning

confidence: 99%

“…More recently, two studies similar to our method have been published by R. Yi et al [22] and K. Nishi et al [21] In the former work, the baseline model is a self-ensemble network, and the sample selection is based on the total classification loss of the model for the original images and transformed images by scaling rotation, and flipping. Our method differs from that method in that it selects images based on the loss distribution of the original image and transforms the selected images with simple patterns by RandAugment.…”

Section: Learning With Noisy Labelsmentioning

confidence: 99%

“…Furthermore, the baseline model of our method is DivideMix. In the latter work, K. Nishi et al [21] proposed AugDesc which utilizes weak augmentation and strong augmentation to images for sample selection and parameter update. The images transformed by the weak augmentation are fed to the model and calculate the loss values for each image.…”

Section: Learning With Noisy Labelsmentioning

confidence: 99%

See 4 more Smart Citations

Consistency Regularization on Clean Samples for Learning with Noisy Labels

Nomura

Kurita

2022

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

In the recent years, deep learning has achieved significant results in various areas of machine learning. Deep learning requires a huge amount of data to train a model, and data collection techniques such as web crawling have been developed. However, there is a risk that these data collection techniques may generate incorrect labels. If a deep learning model for image classification is trained on a dataset with noisy labels, the generalization performance significantly decreases. This problem is called Learning with Noisy Labels (LNL). One of the recent researches on LNL, called DivideMix [1], has successfully divided the dataset into samples with clean labels and ones with noisy labels by modeling loss distribution of all training samples with a two-component Mixture Gaussian model (GMM). Then it treats the divided dataset as labeled and unlabeled samples and trains the classification model in a semi-supervised manner. Since the selected samples have lower loss values and are easy to classify, training models are in a risk of overfitting to the simple pattern during training. To train the classification model without overfitting to the simple patterns, we propose to introduce consistency regularization on the selected samples by GMM. The consistency regularization perturbs input images and encourages model to outputs the same value to the perturbed images and the original images. The classification model simultaneously receives the samples selected as clean and their perturbed ones, and it achieves higher generalization performance with less overfitting to the selected samples. We evaluated our method with synthetically generated noisy labels on CIFAR-10 and CIFAR-100 and obtained results that are comparable or better than the state-of-the-art method.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: Learning With Noisy Labelsmentioning

confidence: 99%

Section: Learning With Noisy Labelsmentioning

confidence: 99%

Section: Learning With Noisy Labelsmentioning

confidence: 99%

See 3 more Smart Citations

Consistency Regularization on Clean Samples for Learning with Noisy Labels

Nomura

Kurita

2022

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

How Does Heterogeneous Label Noise Impact Generalization in Neural Nets?

Khanal

Kanan

2021

Advances in Visual Computing

View full text Add to dashboard Cite

Learning Advisor Networks for Noisy Image Classification

Simone

Uricchio

Bimbo

2022

Image Analysis and Processing – ICIAP 2022

View full text Add to dashboard Cite

In this paper, we introduced the novel concept of advisor network to address the problem of noisy labels in image classification. Deep neural networks (DNN) are prone to performance reduction and overfitting problems on training data with noisy annotations. Weighting loss methods aim to mitigate the influence of noisy labels during the training, completely removing their contribution. This discarding process prevents DNNs from learning wrong associations between images and their correct labels but reduces the amount of data used, especially when most of the samples have noisy labels. Differently, our method weighs the feature extracted directly from the classifier without altering the loss value of each data. The advisor helps to focus only on some part of the information present in mislabeled examples, allowing the classifier to leverage that data as well. We trained it with a meta-learning strategy so that it can adapt throughout the training of the main model. We tested our method on CIFAR10 and CIFAR100 with synthetic noise, and on Clothing1M which contains real-world noise, reporting state-of-the-art results.

show abstract

Augmentation Strategies for Learning with Noisy Labels

Cited by 68 publications

References 10 publications

Consistency Regularization on Clean Samples for Learning with Noisy Labels

Consistency Regularization on Clean Samples for Learning with Noisy Labels

How Does Heterogeneous Label Noise Impact Generalization in Neural Nets?

Learning Advisor Networks for Noisy Image Classification

Contact Info

Product

Resources

About