FedCorr: Multi-Stage Federated Learning for Label Noise Correction

Xu, Jingyi; Chen, Zihan; Quek, Tony Q. S.; Chong, Kai Fong Ernest

doi:10.1109/cvpr52688.2022.00994

Cited by 40 publications

(31 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, in the settings of pair flipping noise and asymmetry noise, the test accuracies of FedLNL are close to that of FedAvg over the clean data (denoted as FedAvg (clean)), indicating that FedLNL nearly achieves the upper bound of its test accuracy. For CIFAR-100, FedLNL improves the test accuracy by up to 25.98% compared with the second best scheme FedCorr (Xu et al 2022).…”

Section: Discussionmentioning

confidence: 98%

“…Most of them need an additional clean dataset to guide the selection, while such a clean dataset is not always available. The second category utilizes a label-correction mechanism to relabel noisy labels, based on the representations extracted from the training data, e.g., the nearest neighbors in the embedding space (Tsouvalas et al 2022) and the prediction of the global model (Xu et al 2022). The third category (e.g., RoFL (Yang et al 2022b) and FedLSR (Jiang et al 2022)) leverages self-supervised learning to obtain more robust representations.…”

Section: Label-noise Learning Schemesmentioning

confidence: 99%

“…These schemes can be classified into the following categories according to the utilized techniques. The first category (e.g., FedCorr (Xu et al 2022)) utilizes a label-correction mechanism to relabel noisy labels. The second category (e.g., RoFL (Yang et al 2022b) and FedLSR (Jiang et al 2022)) leverages self-supervised learning to obtain more robust representations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Federated Label-Noise Learning with Local Diversity Product Regularization

Zhou,

Wang

2024

AAAI

View full text Add to dashboard Cite

Training data in federated learning (FL) frameworks can have label noise, since they must be stored and annotated on clients' devices. If trained over such corrupted data, the models learn the wrong knowledge of label noise, which highly degrades their performance. Although several FL schemes are designed to combat label noise, they suffer performance degradation when the clients' devices only have limited local training samples. To this end, a new scheme called federated label-noise learning (FedLNL) is developed in this paper. The key problem of FedLNL is how to estimate a noise transition matrix (NTM) accurately in the case of limited local training samples. If a gradient-based update method is used to update the local NTM on each client's device, it can generate too large gradients for the local NTM, causing a high estimation error of the local NTM. To tackle this issue, an alternating update method for the local NTM and the local classifier is designed in FedLNL, where the local NTM is updated by a Bayesian inference-based update method. Such an alternating update method makes the loss function of existing NTM-based schemes not applicable to FedLNL. To enable federated optimization of FedLNL, a new regularizer on the parameters of the classifier called local diversity product regularizer is designed for the loss function of FedLNL. The results show that FedLNL improves the test accuracy of a trained model by up to 25.98%, compared with the state-of-the-art FL schemes that tackle label-noise issues.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Label-noise Learning Schemesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Federated Label-Noise Learning with Local Diversity Product Regularization

Zhou,

Wang

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…For example, research has shown that label noise can negatively impact model performance (Ke et al, 2022). Additionally, methods have been proposed to correct labels that have been corrupted by noise (Fang & Ye, 2022;Xu et al, 2022). Recently, Jothimurugesan et al (2022) also investigate the concept shift problem under the assumption that clients do not have concept shifts at the beginning of the training.…”

Section: A Proof Of Em Stepsmentioning

confidence: 99%

FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts

Guo¹,

Tang²,

Lin³

2023

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) is a machine learning paradigm that protects privacy by keeping client data on edge devices. However, optimizing FL in practice can be challenging due to the diversity and heterogeneity of the learning system. Recent research efforts have aimed to improve the optimization of FL with distribution shifts, but it is still an open problem how to train FL models when multiple types of distribution shifts, i.e., feature distribution skew, label distribution skew, and concept shift occur simultaneously. To address this challenge, we propose a novel algorithm framework, FedConceptEM, for handling diverse distribution shifts in FL. FedConceptEM automatically assigns clients with concept shifts to different models, avoiding the performance drop caused by these shifts. At the same time, clients without concept shifts, even with feature or label skew, are assigned to the same model, improving the robustness of the trained models. Extensive experiments demonstrate that FedConceptEM outperforms other state-of-the-art cluster-based FL methods by a significant margin.

show abstract

“…In F-LNL, a global neural network model is fine-tuned via distributed learning across multiple local clients with noisy samples. Like (Xu et al 2022), we here also assume some local clients have noisy labels (namely noisy clients), while others have only clean labels (namely clean clients).…”

Section: Introductionmentioning

confidence: 99%

FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels

Li,

Cheng

et al. 2024

AAAI

View full text Add to dashboard Cite

Federated Learning with Noisy Labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning by aggregating multiple client models trained with local noisy or clean samples. On the basis of a federated learning framework, recent advances primarily adopt label noise filtering to separate clean samples from noisy ones on each client, thereby mitigating the negative impact of label noise. However, these prior methods do not learn noise filters by exploiting knowledge across all clients, leading to sub-optimal and inferior noise filtering performance and thus damaging training stability. In this paper, we present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter for effectively identifying samples with noisy labels on every client, thereby raising stability during local training sessions. Without sacrificing data privacy, this is achieved by modeling the global distribution of label noise across all clients. Then, in an effort to make the global model achieve higher performance, we introduce a Predictive Consistency based Sampler to identify more credible local data for local model training, thus preventing noise memorization and further boosting the training stability. Extensive experiments on CIFAR-10, CIFAR-100, and Clothing1M demonstrate that FedDiv achieves superior performance over state-of-the-art F-LNL methods under different label noise settings for both IID and non-IID data partitions. Source code is publicly available at https://github.com/lijichang/FLNL-FedDiv.

show abstract

FedCorr: Multi-Stage Federated Learning for Label Noise Correction

Cited by 40 publications

References 8 publications

Federated Label-Noise Learning with Local Diversity Product Regularization

Federated Label-Noise Learning with Local Diversity Product Regularization

FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts

FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels

Contact Info

Product

Resources

About