Transfer Learning with Label Noise

Yu, Xiyu; Liu, Tongliang; Gong, Maoguo; Zhang, Kun; Batmanghelich, Kayhan; Tao, Dacheng

doi:10.48550/arxiv.1707.09724

Cited by 16 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the setting of label noise, transition probabilities are introduced to statistically model the generation of noisy labels. In classi cation and transfer learning, methods [25,21,35,38] employ transition probabilities to modify loss functions such that they can be robust to noisy labels. Similar strategies to modify deep neural networks by adding a transition layer have been proposed in [29,26].…”

Section: Related Workmentioning

confidence: 99%

Learning with Biased Complementary Labels

Liu

Gong

et al. 2018

Lecture Notes in Computer Science

Self Cite

133

165

View full text Add to dashboard Cite

In this paper, we study the classi cation problem in which we have access to easily obtainable surrogate for true labels, namely complementary labels, which specify classes that observations do not belong to. Let Y andȲ be the true and complementary labels, respectively. We rst model the annotation of complementary labels via transition probabilities P (Ȳ = i|Y = j), i = j ∈ {1, · · · , c}, where c is the number of classes. Previous methods implicitly assume that P (Ȳ = i|Y = j), ∀i = j, are identical, which is not true in practice because humans are biased toward their own experience. For example, as shown in Figure 1, if an annotator is more familiar with monkeys than prairie dogs when providing complementary labels for meerkats, she is more likely to employ "monkey" as a complementary label. We therefore reason that the transition probabilities will be di erent. In this paper, we propose a framework that contributes three main innovations to learning with biased complementary labels: (1) It estimates transition probabilities with no bias. (2) It provides a general method to modify traditional loss functions and extends standard deep neural network classi ers to learn with biased complementary labels. (3) It theoretically ensures that the classi er learned with complementary labels converges to the optimal one learned with true labels. Comprehensive experiments on several benchmark datasets validate the superiority of our method to current state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning with Biased Complementary Labels

Liu

Gong

et al. 2018

Lecture Notes in Computer Science

Self Cite

133

165

View full text Add to dashboard Cite

show abstract

“…The goal of UDA is to transfer knowledge [21], [22], [23] from an annotated source domain to another unlabeled target domain by reducing domain shift. From the standpoint of feature learning, many UDA studies can be considered as either domain-invariant feature learning or domain-specific feature learning.…”

Section: A Unsupervised Domain Adaptationmentioning

confidence: 99%

Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy

Wang,

Zhang,

Lan

et al. 2020

Preprint

View full text Add to dashboard Cite

Reducing feature redundancy has shown beneficial effects for improving the accuracy of deep learning models, thus it is also indispensable for the models of unsupervised domain adaptation (UDA). Nevertheless, most recent efforts in the field of UDA ignore this point. Moreover, main schemes realizing this in general independent of UDA purely involve a single domain, thus might not be effective for cross-domain tasks. In this paper, we emphasize the significance of reducing feature redundancy for improving UDA in a bi-level way. For the first level, we try to ensure compact domain-specific features with a transferable decorrelated normalization module, which preserves specific domain information whilst easing the side effect of feature redundancy on the sequel domain-invariance. In the second level, domain-invariant feature redundancy caused by domain-shared representation is further mitigated via an alternative brand orthogonality for better generalization. These two novel aspects can be easily plugged into any BN-based backbone neural networks. Specifically, simply applying them to ResNet50 has achieved competitive performance to the state-ofthe-arts on five popular benchmarks. Our code will be available at https://github.com/dreamkily/gUDA.

show abstract

“…The theoretical work on understanding the impact of label noise on the training and generalization of deep neural networks is still ongoing [65]. On the practical side, many studies have shown the negative impact of noisy labels on the performance of these models in real-world applications [66], [67], [68]. Not surprisingly, therefore, this topic has been the subject of much research in recent years.…”

Section: Deep Learning With Noisy Labelsmentioning

confidence: 99%

Deep learning with noisy labels: exploring techniques and remedies in medical image analysis

Karimi¹,

Dou²,

Warfield³

et al. 2019

Preprint

View full text Add to dashboard Cite

Supervised training of deep learning models requires large labeled datasets. There is a growing interest in obtaining such datasets for medical image analysis applications. However, the impact of label noise has not received sufficient attention. Recent studies have shown that label noise can significantly impact the performance of deep learning models in many machine learning and computer vision applications. This is especially concerning for medical applications, where datasets are typically small, labeling requires domain expertise and suffers from high inter-and intra-observer variability, and erroneous predictions may influence decisions that directly impact human health. In this paper, we first review the state-of-the-art in handling label noise in deep learning. Then, we review studies that have dealt with label noise in deep learning for medical image analysis. Our review shows that recent progress on handling label noise in deep learning has gone largely unnoticed by the medical image analysis community. To help achieve a better understanding of the extent of the problem and its potential remedies, we conducted experiments with three medical imaging datasets with different types of label noise. Based on the results of these experiments and our review of the literature, we make recommendations on methods that can be used to alleviate the effects of different types of label noise on deep models trained for medical image analysis. We hope that this article helps the medical image analysis researchers and developers in choosing and devising new techniques that effectively handle label noise in deep learning.

show abstract

Transfer Learning with Label Noise

Cited by 16 publications

References 21 publications

Learning with Biased Complementary Labels

Learning with Biased Complementary Labels

Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy

Deep learning with noisy labels: exploring techniques and remedies in medical image analysis

Contact Info

Product

Resources

About