Federated Deep Reinforcement Learning

Zhuo, Hankz Hankui; Feng, Wenfeng; Lin, Yu-Feng; Xu, Qian; Yang, Qiang

doi:10.48550/arxiv.1901.08277

Cited by 33 publications

(32 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The participating BSs collaboratively train a model based on the local data contents to determine the optimal parameters for a global model. Similarly, the study in [86] utilises DRL under the FL setting to build a highquality model for agents taking into account their privacy. The proposed model is evaluated in two different domains, Text2Action and Grid-world.…”

Section: Network Slicing (Ns)mentioning

confidence: 99%

Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges

Al-Quraan¹,

Mohjazi²,

Bariah³

et al. 2021

Preprint

View full text Add to dashboard Cite

The unprecedented surge of data volume in wireless networks empowered with artificial intelligence (AI) opens up new horizons for providing ubiquitous data-driven intelligent services. Traditional cloud-centric machine learning (ML)-based services are implemented by collecting datasets and training models centrally. However, this conventional training technique encompasses two challenges: (i) high communication and energy cost due to increased data communication, (ii) threatened data privacy by allowing untrusted parties to utilise this information. Recently, in light of these limitations, a new emerging technique, coined as federated learning (FL), arose to bring ML to the edge of wireless networks. FL can extract the benefits of data silos by training a global model in a distributed manner, orchestrated by the FL server. FL exploits both decentralised datasets and computing resources of participating clients to develop a generalised ML model without compromising data privacy. In this article, we introduce a comprehensive survey of the fundamentals and enabling technologies of FL. Moreover, an extensive study is presented detailing various applications of FL in wireless networks and highlighting their challenges and limitations. The efficacy of FL is further explored with emerging prospective beyond fifth generation (B5G) and sixth generation (6G) communication systems. The purpose of this survey is to provide an overview of the state-of-the-art of FL applications in key wireless technologies that will serve as a foundation to establish a firm understanding of the topic. Lastly, we offer a road forward for future research directions.

show abstract

Section: Network Slicing (Ns)mentioning

confidence: 99%

Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges

Al-Quraan¹,

Mohjazi²,

Bariah³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…and number of A at T, n A (T ) increase in non-linear fashion as n(L i ) enlarges linearly. The upper bound of n A (T ) can be defined as (4).…”

Section: B Arithmetic Relation Through Dissecting Clustersmentioning

confidence: 99%

“…However, acquiring IID (Independently and Identically Distributed) data in a designated local machine is not guaranteed in a realworld distributed environment, as most of the existing systems tend to be biased and skewed due to intrinsic propensity and external factors that influence the circumstances. Thus, measures in order to upgrade the FL performance were introduced to successfully mitigate the unstable impact of aggregating non-IID data through approaches such as utilizing learning paradigms [4][5][6][7][8][9], clustering [10][11][12][13][14], knowledge distillation [15][16][17][18], stochastic approach [11], [19][20], Bayesian theorem [21][22][23][24], etc. Initially, FL architecture was specifically designed in order to accommodate the effective training among the distributed conditions without sending the local data itself for security purposes [1], [2], [25], thus many researches were conducted on FL to achieve higher privacy such as detecting adversaries [12], [26].…”

mentioning

confidence: 99%

Robust Convergence in Federated Learning through Label-wise Clustering

Lee¹,

Liu²,

Kim³

et al. 2021

Preprint

View full text Add to dashboard Cite

“…[5] and [14] proposed federated RL algorithms for training policies to maximise content cache-hits on Base Stations at the network edge. [6] designed the FedRL system for training a policy, where individual FL clients do not have access to the full state-space of the RL task. While these works contribute to the development of model-free RL in the FL setting, they suffer from high sample complexity and lack theoretical guarantees.…”

Section: Related Workmentioning

confidence: 99%

“…More recently, federated Reinforcement Learning (RL) was proposed to extend FL to train RL models to solve sequential decision-making problems. The existing federated RL works [5], [6], [7] directly combine model-free RL with FL. Specifically, they train policies locally for all collaborating devices, using the model-free RL objective, and average the policy parameters on the server to generate a global policy for the next round of local training.…”

Section: Introductionmentioning

confidence: 99%

Federated Ensemble Model-based Reinforcement Learning in Edge Computing

Wang¹,

Hu²,

Mills³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a privacy-preserving machine learning paradigm that enables collaborative training among geographically distributed and heterogeneous users without gathering their data. Extending FL beyond the conventional supervised learning paradigm, federated Reinforcement Learning (RL) was proposed to handle sequential decisionmaking problems for various privacy-sensitive applications such as autonomous driving. However, the existing federated RL algorithms directly combine model-free RL with FL, and thus generally have high sample complexity and lack theoretical guarantees. To address the above challenges, we propose a new federated RL algorithm that incorporates model-based RL and ensemble knowledge distillation into FL. Specifically, we utilise FL and knowledge distillation to create an ensemble of dynamics models from clients, and then train the policy by solely using the ensemble model without interacting with the real environment. Furthermore, we theoretically prove that the monotonic improvement of the proposed algorithm is guaranteed. Extensive experimental results demonstrate that our algorithm obtains significantly higher sample efficiency compared to federated model-free RL algorithms in the challenging continuous control benchmark environments. The results also show the impact of non-IID client data and local update steps on the performance of federated RL, validating the insights obtained from our theoretical analysis.

show abstract

Federated Deep Reinforcement Learning

Cited by 33 publications

References 12 publications

Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges

Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges

Robust Convergence in Federated Learning through Label-wise Clustering

Federated Ensemble Model-based Reinforcement Learning in Edge Computing

Contact Info

Product

Resources

About