Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

Liu, Boyi; Wang, Lujia; Liu, Ming

doi:10.1109/iros40897.2019.8967908

Cited by 67 publications

(65 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Network protocols are not entirely new. They have been around since the 1970s and 1980s, with these network protocols being modified over time to reflect current trends [90]. In the context of Federated Learning, there have been proposed network protocols and methods to tackle a few issues with FL, mainly related to privacy and traffic flow [91].…”

Section: Secure Network Protocols In Support Of Federated Learningmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

This paper provides a comprehensive study of Federated Learning (FL) with an emphasis on enabling software and hardware platforms, protocols, real-life applications and use-cases. FL can be applicable to multiple domains but applying it to different industries has its own set of obstacles. FL is known as collaborative learning, where algorithm(s) get trained across multiple devices or servers with decentralized data samples without having to exchange the actual data. This approach is radically different from other more established techniques such as getting the data samples uploaded to servers or having data in some form of distributed infrastructure. FL on the other hand generates more robust models without sharing data, leading to privacy-preserved solutions with higher security and access privileges to data. This paper starts by providing an overview of FL. Then, it gives an overview of technical details that pertain to FL enabling technologies, protocols, and applications. Compared to other survey papers in the field, our objective is to provide a more thorough summary of the most relevant protocols, platforms, and real-life use-cases of FL to enable data scientists to build better privacy-preserving solutions for industries in critical need of FL. We also provide an overview of key challenges presented in the recent literature and provide a summary of related research work. Moreover, we explore both the challenges and advantages of FL and present detailed service use-cases to illustrate how different architectures and protocols that use FL can fit together to deliver desired results.

show abstract

Section: Secure Network Protocols In Support Of Federated Learningmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

show abstract

“…To demonstrate the effectiveness of the proposed architecture, a non-player character in the Atari game Pong is implemented and evaluated. In [57], the authors propose the Lifelong Federated Reinforcement Learning (LFRL) for navigation in cloud robotic systems. It enables the robot to learn efficiently in a new environment and use prior knowledge to quickly adapt to the changes in the environment.…”

Section: Horizontal Federated Reinforcement Learningmentioning

confidence: 99%

“…Automated control of robots is a typical example of control optimization problems. [57] discusses robot navigation scenarios and focuses on how to make robots transfer their experience so that they can make use of prior knowledge and quickly adapt to changing environments. As a solution, a cooperative learning architecture, called LFRL, is proposed for navigation in cloud robotic systems.…”

Section: Frl For Control Optimizationmentioning

confidence: 99%

“…Participants in other similar environments can continue to use FRL for further updating their parameters to achieve the desired model performance according to their individual environments. The paper [57] also suggests that the sharing global model in the cloud is not the final policy model for local participants. An effective transfer learning should be applied to resolve the structural difference between the shared network and private network.…”

Section: Lessons Learned From the Aggregation Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Federated reinforcement learning: techniques, applications, and open challenges

Qi¹,

Zhou²,

Lei³

et al. 2021

View full text Add to dashboard Cite

This paper presents a comprehensive survey of Federated Reinforcement Learning (FRL), an emerging and promising field in Reinforcement Learning (RL). Starting with a tutorial of Federated Learning (FL) and RL, we then focus on the introduction of FRL as a new method with great potential by leveraging the basic idea of FL to improve the performance of RL while preserving data-privacy. According to the distribution characteristics of the agents in the framework, FRL algorithms can be divided into two categories, i.e., Horizontal Federated Reinforcement Learning (HFRL) and Vertical Federated Reinforcement Learning (VFRL). We provide the detailed definitions of each category by formulas, investigate the evolution of FRL from a technical perspective, and highlight its advantages over previous RL algorithms. In addition, the existing works on FRL are summarized by application fields, including edge computing, communication, control optimization, and attack detection. Finally, we describe and discuss several key research directions that are crucial to solving the open problems within FRL.

show abstract

“…Deep Q-Networks (DQN)algorithms are most frequently adopted in AIOT systems in recent years. A few applications of DQN and Double DQN (DDQN) in IoT communications systems are: [315], [316], [317], [318]and [319]; in IoT Cloud/Fog/Edge computing are: [320], [321], [322], [323] and [324]; in autonomous IoT robotics are: [325], [326], [327], [328] and [329] ; in IoT smart vehicles are: [330], [331] and [299] and in smart grids are: [332], [333], [334] and [335] respectively.…”

Section: E Autonomous Iotmentioning

confidence: 99%

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Naeem

Rizvi

Coronato³

2020

IEEE Access

126

View full text Add to dashboard Cite

Due to the recent progress in Deep Neural Networks, Reinforcement Learning (RL) has become one of the most important and useful technology. It is a learning method where a software agent interacts with an unknown environment, selects actions, and progressively discovers the environment dynamics. RL has been effectively applied in many important areas of real life. This article intends to provide an in-depth introduction of the Markov Decision Process, RL and its algorithms. Moreover, we present a literature review of the application of RL to a variety of fields, including robotics and autonomous control, communication and networking, natural language processing, games and self-organized system, scheduling management and configuration of resources, and computer vision.

show abstract

Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

Cited by 67 publications

References 12 publications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated reinforcement learning: techniques, applications, and open challenges

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Contact Info

Product

Resources

About