Approaches to address the data skew problem in federated learning

Verma, Dinesh; White, Graham; Julier, Simon; Pasteris, Stepehen; Chakraborty, Supriyo; Cirincione, Greg

doi:10.1117/12.2519621

Cited by 22 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In these scenarios, it would not be beneficial to have a naive implementation of an FL model because the resulting model would be inaccurate [105]. To remedy the data skewing problem, we would need to rely on the server [106]. Furthermore, data skewing can even affect different kinds of data that can be used in FL.…”

Section: Challenges and Limitations Of Federated Learningmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

This paper provides a comprehensive study of Federated Learning (FL) with an emphasis on enabling software and hardware platforms, protocols, real-life applications and use-cases. FL can be applicable to multiple domains but applying it to different industries has its own set of obstacles. FL is known as collaborative learning, where algorithm(s) get trained across multiple devices or servers with decentralized data samples without having to exchange the actual data. This approach is radically different from other more established techniques such as getting the data samples uploaded to servers or having data in some form of distributed infrastructure. FL on the other hand generates more robust models without sharing data, leading to privacy-preserved solutions with higher security and access privileges to data. This paper starts by providing an overview of FL. Then, it gives an overview of technical details that pertain to FL enabling technologies, protocols, and applications. Compared to other survey papers in the field, our objective is to provide a more thorough summary of the most relevant protocols, platforms, and real-life use-cases of FL to enable data scientists to build better privacy-preserving solutions for industries in critical need of FL. We also provide an overview of key challenges presented in the recent literature and provide a summary of related research work. Moreover, we explore both the challenges and advantages of FL and present detailed service use-cases to illustrate how different architectures and protocols that use FL can fit together to deliver desired results.

show abstract

Section: Challenges and Limitations Of Federated Learningmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Since each local device records the activities of its owner, data across devices tend to have different sizes, features, and target classes distribution. This technically means that the local data of one single client cannot be considered to be representative of the overall data distribution [78]. Three scenarios of non-IID data in federated learning can be encountered, i.e., class imbalance, distribution imbalance and size imbalance.…”

Section: Non Independent and Identically Distributed Datamentioning

confidence: 99%

Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems

Wahab

Mourad

Otrok

et al. 2021

IEEE Commun. Surv. Tutorials

345

110

View full text Add to dashboard Cite

The communication and networking field is hungry for machine learning decision-making solutions to replace the traditional model-driven approaches that proved to be not rich enough for seizing the ever-growing complexity and heterogeneity of the modern systems in the field. Traditional machine learning solutions assume the existence of (cloud-based) central entities that are in charge of processing the data. Nonetheless, the difficulty of accessing private data, together with the high cost of transmitting raw data to the central entity gave rise to a decentralized machine learning approach called Federated Learning. The main idea of federated learning is to perform an on-device collaborative training of a single machine learning model without having to share the raw training data with any third-party entity. Although few survey articles on federated learning already exist in the literature, the motivation of this survey stems from three essential observations. The first one is the lack of a fine-grained multi-level classification of the federated learning literature, where the existing surveys base their classification on only one criterion or aspect. The second observation is that the existing surveys focus only on some common challenges, but disregard other essential aspects such as reliable client selection, resource management and training service pricing. The third observation is the lack of explicit and straightforward directives for researchers to help them design future federated learning solutions that overcome the state-of-the-art research gaps. To address these points, we first provide a comprehensive tutorial on federated learning and its associated concepts, technologies and learning approaches. We then survey and highlight the applications and future directions of federated learning in the domain of communication and networking. Thereafter, we design a three-level classification scheme that first categorizes the federated learning literature based on the high-level challenge that they tackle. Then, we classify each high-level challenge into a set of specific low-level challenges to foster a better understanding of the topic. Finally, we provide, within each low-level challenge, a fine-grained classification based on the technique used to address this particular challenge. For each category of high-level challenges, we provide a set of desirable criteria and future research directions that are aimed to help the research community design innovative and efficient future solutions. To the best of our knowledge, our survey is the most comprehensive in terms of challenges and techniques it covers and the most fine-grained in terms of the multi-level classification scheme it presents.

show abstract

“…For training data that consists of features (categories I and III in Table I), the data collected at different sites may correspond to different regions of the feature space. Extrapolating across different regions of applicability can cause mistakes in the estimation of the AI model [9]. Different sites may also have collected different volumes of training data, which may make some trained models to be better than others.…”

Section: Challenges In Federated Learningmentioning

confidence: 99%

Federated Learning for Coalition Operations

Verma,

Calo,

Witherspoon

et al. 2019

Preprint

View full text Add to dashboard Cite

Approaches to address the data skew problem in federated learning

Cited by 22 publications

References 8 publications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems

Federated Learning for Coalition Operations

Contact Info

Product

Resources

About