Jo-SRC: A Contrastive Approach for Combating Noisy Labels

Yao, Yazhou; Sun, Zeren; Zhang, Chuanyi; Shen, Fumin; Wu, Qi; Zhang, Jian; Tang, Zhenmin

doi:10.1109/cvpr46437.2021.00515

Cited by 112 publications

(71 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For FL, we select four algorithms covering representative FL strategies related to FedNoiL: (i) FedAvg [24], (ii) personalized FL: APFL [8], (iii) robust FL: Krum [5], and CFL [30]. For NLL, we consider two SoTA methods: (i) DMix [20] and (ii) JoSRC [40]. We denote these baselines in the form of "FL" + "NLL".…”

Section: Methodsmentioning

confidence: 99%

“…Co-teaching [12] lets two networks to select clean training data for each other. JoCoR [36] selects small-loss (cross-entropy and co-regularization loss) samples as clean data, while JoSRC [40] selects data based on Jensen-Shannon divergence. Another line of works [20,25] applies semi-supervised learning [4,32] that treats wrongly-labeled samples as unlabeled and assigns them with pseudo labels.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels

Wang¹,

Zhou²,

Long³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Hence, the labels in practice are usually annotated by clients of varying expertise or criteria and thus contain different amounts of noises. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. Although recent robust FL methods take malicious clients into account, they have not addressed local noisy labels on each device and the impact on the global model. In this paper, we develop a simple two-level sampling method "FedNoiL" that (1) selects clients for more robust global aggregation on the server; and (2) selects clean labels and correct pseudo-labels at the client end for more robust local training. The sampling probabilities are built upon clean label detection by the global model. Moreover, we investigate different schedules changing the local epochs between aggregations over the course of FL, which notably improves the communication and computation efficiency in noisy label setting. In experiments with homogeneous/heterogeneous data distributions and noise ratios, we observed that direct combinations of SOTA FL methods with SOTA noisy-label learning methods can easily fail but our method consistently achieves better and robust performance.Preprint. Under review.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels

Wang¹,

Zhou²,

Long³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Acc. Standard CE 84.03 CleanNet [16] 83.95 Decoupling [23] 85.53 Co-teaching [9] 61.91 Co-teaching+ [43] 81.61 JoCoR [36] 77.94 Jo-SRC [41] 86.66 Co-learning [30] 87.57 Tripartite 88.34…”

Section: Food-101n Methodsmentioning

confidence: 99%

“…Accuracy comparison on the Food-101N. The result of Jo-SRC is from [41], the rest of results are from [30].…”

Section: Food-101n Methodsmentioning

confidence: 99%

Tripartite: Tackle Noisy Labels by a More Precise Partition

Liang¹,

Yao²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Samples in large-scale datasets may be mislabeled due to various reasons, and Deep Neural Networks can easily over-fit to the noisy label data. To tackle this problem, the key point is to alleviate the harm of these noisy labels. Many existing methods try to divide training data into clean and noisy subsets in terms of loss values, and then process the noisy label data variedly. One of the reasons hindering a better performance is the hard samples. As hard samples always have relatively large losses whether their labels are clean or noisy, these methods could not divide them precisely. Instead, we propose a Tripartite solution to partition training data more precisely into three subsets: hard, noisy, and clean. The partition criteria are based on the inconsistent predictions of two networks, and the inconsistency between the prediction of a network and the given label. To minimize the harm of noisy labels but maximize the value of noisy label data, we apply a low-weight learning on hard data and a self-supervised learning on noisy label data without using the given labels. Extensive experiments demonstrate that Tripartite can filter out noisy label data more precisely, and outperforms most state-of-the-art methods on five benchmark datasets, especially on real-world datasets.

show abstract

“…This problem is even more crucial in the medical field, given that the annotation quality requires great expertise. Therefore, understanding, modeling, and learning with noisy labels has gained great momentum in recent research efforts [5,23,8,17,20,11,28,40,16,34,47,41,49,36,48].…”

Section: Introductionmentioning

confidence: 99%

Learning to Bootstrap for Combating Label Noise

Zhou¹,

Li²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep neural networks are powerful tools for representation learning, but can easily overfit to noisy labels which are prevalent in many realworld scenarios. Generally, noisy supervision could stem from variation among labelers, label corruption by adversaries, etc. To combat such label noises, one popular line of approach is to apply customized weights to the training instances, so that the corrupted examples contribute less to the model learning. However, such learning mechanisms potentially erase important information about the data distribution and therefore yield suboptimal results. To leverage useful information from the corrupted instances, an alternative is the bootstrapping loss, which reconstructs new training targets on-the-fly by incorporating the network's own predictions (i.e., pseudo-labels).In this paper, we propose a more generic learnable loss objective which enables a joint reweighting of instances and labels at once. Specifically, our method dynamically adjusts the per-sample importance weight between the real observed labels and pseudo-labels, where the weights are efficiently determined in a meta process. Compared to the previous instance reweighting methods, our approach concurrently conducts implicit relabeling, and thereby yield substantial improvements with almost no extra cost. Extensive experimental results demonstrated the strengths of our approach over existing methods on multiple natural and medical image benchmark datasets, including CIFAR-10, CIFAR-100, ISIC2019 and Clothing 1M. The code is publicly available at https://github.com/yuyinzhou/L2B.

show abstract

Jo-SRC: A Contrastive Approach for Combating Noisy Labels

Cited by 112 publications

References 26 publications

FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels

FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels

Tripartite: Tackle Noisy Labels by a More Precise Partition

Learning to Bootstrap for Combating Label Noise

Contact Info

Product

Resources

About