Fast-Convergent Federated Learning

Nguyen, Hung T.; Sehwag, Vikash; Hosseinalipour, Seyyedali; Brinton, Christopher G.; Chiang, Mung; Poor, H. Vincent

doi:10.1109/jsac.2020.3036952

Cited by 164 publications

(67 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, without considering this issue at the time of updating the global model, the updated global model will be biased towards local models of devices that are closer to the server. Existing works [35,36,37,38,39,40] assume that transmissions are always successful, which is not correct in reality.…”

Section: Problemmentioning

confidence: 99%

“…However, it was assumed that all of the devices participate in the aggregation step, which is obviously not possible when the number of available resource blocks is limited. To tackle this problem, in [35,36,37,38,39,40] at the beginning of each round, the server samples a subset of devices and allocates the available resource blocks to these devices. After performing local update steps, the BS aggregates the local models of the chosen (scheduled) devices and updates the global model.…”

Section: Related Workmentioning

confidence: 99%

“…After performing local update steps, the BS aggregates the local models of the chosen (scheduled) devices and updates the global model. The works in [35,36,37,38,39,40] assumed that the BS successfully receives the local models of all the scheduled devices, and they designed the global model update step only based on the scheduling policy. However, in reality, not only is the success probability less than one, but also it is different for different devices.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks

Salehi

Hossain

2021

IEEE Trans. Commun.

View full text Add to dashboard Cite

Use cases of future wireless networks (e.g. fifth-generation [5G] networks and beyond [B5G]) will have servicequality requirements including higher data rates than today's networks for enhanced mobile broadband (eMBB), minimal latency and high network availability for ultra-reliability low-latency connection (URLLC), and massive access support for machine-type communications (mMTC). Also, 5G and B5G are expected to support communications for highly mobile scenarios with applications in new vertical sectors such as unmanned aerial vehicle (UAV) and autonomous car. Therefore, 5G and B5G cellular systems require a set of new technology enablers and solutions. In this thesis, we address some of the challenges of future wireless networks. In particular, we develop novel analytical models as well as methods, which will enable us to obtain insights into the performance of large-scale cellular networks and optimize network parameters.Non-orthogonal multiple access (NOMA) is a promising multiple access technique that enables massive connectivity and reduces the delay. We develop an analytical framework to derive the distribution of transmission success probabilities, meta distribution, for uplink and downlink NOMA. We also investigate the accuracy of distance-based ranking, instead of instantaneous signal power-based ranking, in the successive interference cancellation (SIC) at the NOMA receiver.Sojourn time, the time duration that a mobile user stays within a cell, is a mobility-aware parameter that can significantly impact the performance of mobile users and it can also be exploited to improve resource allocation and mobility management methods in the network. We derive the distribution and mean of the sojourn time in multi-tier cellular networks.Future wireless networks will exploit data-driven machine learning techniques for improving network management as well as service provisioning. Due to privacy and communication issues, learning at a centralized location (for example, at a base station) by collecting data from the mobile devices may not be always feasible. Federated learning is a machine learning setting where the centralized location trains a learning model using remote devices. Federated learning algorithms cannot be employed in real-world scenarios unless they consider unreliable and resource-constrained nature of the wireless medium. We propose a federated learning algorithm that is suitable for wireless networks.

show abstract

Section: Problemmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks

Salehi

Hossain

2021

IEEE Trans. Commun.

View full text Add to dashboard Cite

show abstract

“…Beyond systems heterogeneity of the FL clients, statistical heterogeneity or divergence of client model update is also a concern in federated networks. Some recent FL works (Dinh et al, 2019;Wang et al, 2019;Chen et al, 2020a;Guo et al, 2020;Nguyen et al, 2020) analyze how to guarantee convergence both in theoretically and empirically for an FL setting. The major problem is that they assume all FL clients are resource capable to perform a predefined uniform number of iterations while considering all the devices to participate in the training round.…”

Section: Background and Related Workmentioning

confidence: 99%

FedPARL: Client Activity and Resource-Oriented Lightweight Federated Learning Model for Resource-Constrained Heterogeneous IoT Environment

Imteaj

Amini

2021

Front. Comms. Net.

View full text Add to dashboard Cite

Federated Learning (FL) is a recently invented distributed machine learning technique that allows available network clients to perform model training at the edge, rather than sharing it with a centralized server. Unlike conventional distributed machine learning approaches, the hallmark feature of FL is to allow performing local computation and model generation on the client side, ultimately protecting sensitive information. Most of the existing FL approaches assume that each FL client has sufficient computational resources and can accomplish a given task without facing any resource-related issues. However, if we consider FL for a heterogeneous Internet of Things (IoT) environment, a major portion of the FL clients may face low resource availability (e.g., lower computational power, limited bandwidth, and battery life). Consequently, the resource-constrained FL clients may give a very slow response, or may be unable to execute expected number of local iterations. Further, any FL client can inject inappropriate model during a training phase that can prolong convergence time and waste resources of all the network clients. In this paper, we propose a novel tri-layer FL scheme, Federated Proximal, Activity and Resource-Aware 31 Lightweight model (FedPARL), that reduces model size by performing sample-based pruning, avoids misbehaved clients by examining their trust score, and allows partial amount of work by considering their resource-availability. The pruning mechanism is particularly useful while dealing with resource-constrained FL-based IoT (FL-IoT) clients. In this scenario, the lightweight training model will consume less amount of resources to accomplish a target convergence. We evaluate each interested client's resource-availability before assigning a task, monitor their activities, and update their trust scores based on their previous performance. To tackle system and statistical heterogeneities, we adapt a re-parameterization and generalization of the current state-of-the-art Federated Averaging (FedAvg) algorithm. The modification of FedAvg algorithm allows clients to perform variable or partial amounts of work considering their resource-constraints. We demonstrate that simultaneously adapting the coupling of pruning, resource and activity awareness, and re-parameterization of FedAvg algorithm leads to more robust convergence of FL in IoT environment.

show abstract

“…Reference [8] broadens the previous work to include non-IID data, and they allow the server to act as one of the agents; the server collects from the selected agents some of their data and runs SGD on them. In [9], a non-uninform sampling scheme of agents is considered; the probability distribution of the sampling of agents is calculated by maximizing the inner product between the global and local gradients. They provide an approximate solution, since the actual calculations of the sampling probabilities is non-trivial.…”

Section: Introduction and Related Materialsmentioning

confidence: 99%

Optimal Importance Sampling for Federated Learning

Rizk

Vlaski

Sayed

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Federated learning involves a mixture of centralized and decentralized processing tasks, where a server regularly selects a sample of the agents and these in turn sample their local data to compute stochastic gradients for their learning updates. The sampling of both agents and data is generally uniform; however, in this work we consider non-uniform sampling. We derive optimal importance sampling strategies for both agent and data selection and show that under convexity and Lipschitz assumptions, non-uniform sampling without replacement improves the performance of the original FedAvg algorithm. We run experiments on a regression and classification problem to illustrate the theoretical results.

show abstract

Fast-Convergent Federated Learning

Cited by 164 publications

References 11 publications

Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks

Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks

FedPARL: Client Activity and Resource-Oriented Lightweight Federated Learning Model for Resource-Constrained Heterogeneous IoT Environment

Optimal Importance Sampling for Federated Learning

Contact Info

Product

Resources

About