Federated Learning: Challenges, Methods, and Future Directions

Li, Tian; Sahu, Anit Kumar; Talwalkar, Ameet; Smith, Virginia

doi:10.1109/msp.2020.2975749

Cited by 2,823 publications

(1,460 citation statements)

References 16 publications

Supporting

Mentioning

1,453

Contrasting

Unclassified

Order By: Relevance

“…On top of these methods, one can further enhance communication efficiency, by additionally quantizing gradients [191], removing insignificant gradients, i.e., sparsification [192], opportunistic uploading based on gradient magnitudes [193], and adaptively adjusting the communication intervals [31], [194]. For more details on state dimensionality reduction and advanced FL and FD frameworks in both supervised learning and RL, the readers are encouraged to check [186], [195], and [196].…”

Section: ) Each Head Device Updates Its Primal Variables Asmentioning

confidence: 99%

Wireless Network Intelligence at the Edge

et al. 2019

View full text Add to dashboard Cite

edge devices. The new breed of intelligent devices and high-stake applications (drones, augmented/virtual reality, autonomous systems, and so on) requires a novel paradigm change calling for distributed, low-latency and reliable ML at the wireless network edge (referred to as edge ML). In edge ML, training data are unevenly distributed over a large number of edge nodes, which have access to a tiny fraction of the data. Moreover, training and inference are carried out collectively over wireless links, where edge devices communicate and exchange their learned models (not their private data). In a first of its kind, this article explores the key building blocks of edge ML, different neural network architectural splits and their inherent tradeoffs, as well as theoretical and technical enablers stemming from a wide range of mathematical disciplines. Finally, several case studies pertaining to various high-stake applications are presented to demonstrate the effectiveness of edge ML in unlocking the full potential of 5G and beyond.

show abstract

Section: ) Each Head Device Updates Its Primal Variables Asmentioning

confidence: 99%

Wireless Network Intelligence at the Edge

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The growing computational power of edge devices allows us to leave the data decentralized and push the network computation to the client, which is also ideal from a privacy aspect. The expanding area of federated learning [20], [37], [38] explores developing methods to achieve the goal of learning from highly distributed and heterogeneous data through aggregating locally trained models on remote devices, such as smartphones and wearables. In this case, the intention is to minimize the following objective [37]:…”

Section: Federated Learningmentioning

confidence: 99%

“…To produce a global model, Federated Averaging algorithm [20] is typically used to accumulate client updates after every round of local training t as with Equation 3. The research interest in federated learning revolves around improving communication efficiency [39], personalization [40], fault tolerance [41], privacy preservation [42] as well as looking into the theoretical underpinning of the federated optimization [37]. Specifically, the recent work deals with learning a unified model to solve a single as well as multiple tasks [43].…”

Section: Federated Learningmentioning

confidence: 99%

Federated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence

Saeed

Salim

Özçelebi

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models due to privacy, bandwidth limitations, and the prohibitive cost of annotations. Federated learning provides a compelling framework for learning models from decentralized data, but conventionally, it assumes the availability of labeled samples, whereas on-device data are generally either unlabeled or cannot be annotated readily through user interaction. To address these issues, we propose a self-supervised approach termed scalogramsignal correspondence learning based on wavelet transform to learn useful representations from unlabeled sensor inputs, such as electroencephalography, blood volume pulse, accelerometer, and WiFi channel state information. Our auxiliary task requires a deep temporal neural network to determine if a given pair of a signal and its complementary viewpoint (i.e., a scalogram generated with a wavelet transform) align with each other or not through optimizing a contrastive objective. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains. We demonstrate the effectiveness of representations learned from an unlabeled input collection on downstream tasks with training a linear classifier over pretrained network, usefulness in low-data regime, transfer learning, and cross-validation. Our methodology achieves competitive performance with fullysupervised networks, and it outperforms pre-training with autoencoders in both central and federated contexts. Notably, it improves the generalization in a semi-supervised setting as it reduces the volume of labeled data required through leveraging self-supervised learning.

show abstract

“…Federated Learning (FL) is a DAI model where multi-agents collaboratively share their local knowledge for faster convergence and to make better decisions. FL concept, applications, challenges, and methods have been reviewed in [ 38 , 39 ]. Smith and Hollinger [ 40 ] developed a distributed robotic system that collaboratively shares their knowledge for a single goal (environment exploration).…”

Section: Background and Related Workmentioning

confidence: 99%

Distributed Artificial Intelligence-as-a-Service (DAIaaS) for Smarter IoE and 6G Environments

Janbi

Katib

Albeshri

et al. 2020

Sensors

View full text Add to dashboard Cite

Artificial intelligence (AI) has taken us by storm, helping us to make decisions in everything we do, even in finding our “true love” and the “significant other”. While 5G promises us high-speed mobile internet, 6G pledges to support ubiquitous AI services through next-generation softwarization, heterogeneity, and configurability of networks. The work on 6G is in its infancy and requires the community to conceptualize and develop its design, implementation, deployment, and use cases. Towards this end, this paper proposes a framework for Distributed AI as a Service (DAIaaS) provisioning for Internet of Everything (IoE) and 6G environments. The AI service is “distributed” because the actual training and inference computations are divided into smaller, concurrent, computations suited to the level and capacity of resources available with cloud, fog, and edge layers. Multiple DAIaaS provisioning configurations for distributed training and inference are proposed to investigate the design choices and performance bottlenecks of DAIaaS. Specifically, we have developed three case studies (e.g., smart airport) with eight scenarios (e.g., federated learning) comprising nine applications and AI delivery models (smart surveillance, etc.) and 50 distinct sensor and software modules (e.g., object tracker). The evaluation of the case studies and the DAIaaS framework is reported in terms of end-to-end delay, network usage, energy consumption, and financial savings with recommendations to achieve higher performance. DAIaaS will facilitate standardization of distributed AI provisioning, allow developers to focus on the domain-specific details without worrying about distributed training and inference, and help systemize the mass-production of technologies for smarter environments.

show abstract

Federated Learning: Challenges, Methods, and Future Directions

Cited by 2,823 publications

References 16 publications

Wireless Network Intelligence at the Edge

Wireless Network Intelligence at the Edge

Federated Self-Supervised Learning of Multisensor Representations for Embedded Intelligence

Distributed Artificial Intelligence-as-a-Service (DAIaaS) for Smarter IoE and 6G Environments

Contact Info

Product

Resources

About