BACombo—Bandwidth-Aware Decentralized Federated Learning

Jiang, Jingyan; Hu, Liang; Hu, Chenghao; Liu, Jiate; Wang, Zhi

doi:10.3390/electronics9030440

Cited by 36 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, their design also requires a fully connected network topology and a prerequisite of randomly distributed data among the workers. While in [27], the authors further extend the Combo into a bandwidth aware solution BACombo by greedily choosing the bandwidth-sufficient worker to reduce the transmission delay. In [28], the authors design an experimental study to compare federated learning with gossip learning, and find that gossip learning is comparable to federated learning in their result.…”

Section: Decentralized Federated Learning Implementationmentioning

confidence: 99%

DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology

Chen¹,

Li²,

Zhu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a burgeoning distributed machine learning framework where a central parameter server (PS) coordinates many local users to train a globally consistent model. Conventional federated learning inevitably relies on a centralized topology with a PS. As a result, it will paralyze once the PS fails. To alleviate such a single point failure, especially on the PS, some existing work has provided decentralized FL (DFL) implementations like CDSGD and D-PSGD to facilitate FL in a decentralized topology. However, there are still some problems with these methods, e.g., significant divergence between users' final models in CDSGD and a network-wide model average necessity in D-PSGD. In order to solve these deficiency, this paper devises a new DFL implementation coined as DACFL, where each user trains its model using its own training data and exchanges the intermediate models with its neighbors through a symmetric and doubly stochastic matrix. The DACFL treats the progress of each user's local training as a discrete-time process and employs a first order dynamic average consensus (FODAC) method to track the average model in the absence of the PS. In this paper, we also provide a theoretical convergence analysis of DACFL on the premise of i.i.d data to strengthen its rationality. The experimental results on MNIST, Fashion-MNIST and CIFAR-10 validate the feasibility of our solution in both time-invariant and time-varying network topologies, and declare that DACFL outperforms D-PSGD and CDSGD in most cases.

show abstract

Section: Decentralized Federated Learning Implementationmentioning

confidence: 99%

DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology

Chen¹,

Li²,

Zhu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, it suffers slow convergence when the training data is not identically distributed [23] and the centralised architecture will bring the network congestion in the global server. To reduce the network congestion, the decentralised methods [11, 12, 24] are proposed, every device is connected over WAN, and there is no global server to aggregate the updates. The former work exchanged the partial gradients with all the other devices, while the method in [12] exchanges the partial model weights with a subset of total devices.…”

Section: Related Workmentioning

confidence: 99%

“…Jiang et al . [24] further proposed a bandwidth‐aware device selection method to reduce the communication latency. Although their improvement in communication efficiency, these methods are based on the local updating (SGD) and simply global averaging, leading to the limits of the convergence rate of SGD.…”

Section: Related Workmentioning

confidence: 99%

“…The former work exchanged the partial gradients with all the other devices, while the method in [12] exchanges the partial model weights with a subset of total devices. Jiang et al [ 24] further proposed a bandwidth-aware device selection method to reduce the communication latency. Although their improvement in communication efficiency, these methods are based on the local updating (SGD) and simply global averaging, leading to the limits of the convergence rate of SGD.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Decentralised federated learning with adaptive partial gradient aggregation

Jiang

2020

CAAI trans. intell. technol.

Self Cite

View full text Add to dashboard Cite

Federated learning aims to collaboratively train a machine learning model with possibly geo‐distributed workers, which is inherently communication constrained. To achieve communication efficiency, the conventional federated learning algorithms allow the worker to decrease the communication frequency by training the model locally for multiple times. Conventional federated learning architecture, inherited from the parameter server design, relies on highly centralised topologies and large nodes‐to‐server bandwidths, and convergence property relies on the stochastic gradient descent training in local, which usually causes the large end‐to‐end training latency in real‐world federated learning scenarios. Thus, in this study, the authors propose the adaptive partial gradient aggregation method, a gradient partial level decentralised federated learning, to tackle this problem. In FedPGA, they propose a partial gradient exchange mechanism that makes full use of node‐to‐node bandwidth for speeding up the communication time. Besides, an adaptive model updating method further reduces the convergence rate by adaptive increasing the step size of the stable direction of gradient descent. The experimental results on various datasets demonstrate that the training time is reduced up to 14× compared to baselines without accuracy degrade.

show abstract

“…The recent development of machine learning evolves towards the distributed environment. One of the promising solutions is so-called federated learning that trains user data on the device and exchanges only training models and their update information [5][6][7]. In federated learning, the central server computes only the aggregated average of the training data gathered from each device.…”

Section: Introductionmentioning

confidence: 99%

Internet Traffic Classification with Federated Learning

Mun

Lee

2020

Electronics

View full text Add to dashboard Cite

As Internet traffic classification is a typical problem for ISPs or mobile carriers, there have been a lot of studies based on statistical packet header information, deep packet inspection, or machine learning. Due to recent advances in end-to-end encryption and dynamic port policies, machine or deep learning has been an essential key to improve the accuracy of packet classification. In addition, ISPs or mobile carriers should carefully deal with the privacy issue while collecting user packets for accounting or security. The recent development of distributed machine learning, called federated learning, collaboratively carries out machine learning jobs on the clients without uploading data to a central server. Although federated learning provides an on-device learning framework towards user privacy protection, its feasibility and performance of Internet traffic classification have not been fully examined. In this paper, we propose a federated-learning traffic classification protocol (FLIC), which can achieve an accuracy comparable to centralized deep learning for Internet application identification without privacy leakage. FLIC can classify new applications on-the-fly when a participant joins in learning with a new application, which has not been done in previous works. By implementing the prototype of FLIC clients and a server with TensorFlow, the clients gather packets, perform the on-device training job and exchange the training results with the FLIC server. In addition, we demonstrate that federated learning-based packet classification achieves an accuracy of 88% under non-independent and identically distributed (non-IID) traffic across clients. When a new application that can be classified dynamically as a client participates in learning was added, an accuracy of 92% was achieved.

show abstract

BACombo—Bandwidth-Aware Decentralized Federated Learning

Cited by 36 publications

References 12 publications

DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology

DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology

Decentralised federated learning with adaptive partial gradient aggregation

Internet Traffic Classification with Federated Learning

Contact Info

Product

Resources

About