Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

Liu, Boyi; Wang, Lujia; Liu, Ming

doi:10.1109/lra.2019.2931179

Cited by 164 publications

(39 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cloud robotics applications include perception and computer vision applications, navigation, grasping or manipulation, manufacturing or service robotics, etc.. In [63], an effective transfer learning scheme based on lifelong federated reinforcement learning (LFRL) is proposed for the navigation in cloud robotic systems, where the robots can effectively use prior knowledge and quickly adapt to new environments. The authors in [64] propose an RL-based resource allocation scheme, which can help the cloud to decide whether a request should be accepted and how many resources are supposed to be allocated.…”

Section: A Perception Layer -Autonomous Robotsmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Lei

Tan

Zheng

et al. 2020

IEEE Commun. Surv. Tutorials

214

View full text Add to dashboard Cite

The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, which collect and share information to reflect the status of physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept -autonomous IoT (AIoT). The sensors collect information on the system status, based on which intelligent agents in IoT devices as well as Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide comprehensive survey of the state-of-art research, and then propose a general model for the applications of RL/DRL in AIoT. Finally, the challenges and open issues for future research are identified.

show abstract

Section: A Perception Layer -Autonomous Robotsmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Lei

Tan

Zheng

et al. 2020

IEEE Commun. Surv. Tutorials

214

View full text Add to dashboard Cite

show abstract

“…This method is based on ORB-SLAM2 which adds depth constraint when tracking. We are also interested in cloud robotic systems [23], [24], [25], [26] and we will apply our work to cloud robots in the future.…”

Section: Related Workmentioning

confidence: 99%

A Robust Stereo Camera Localization Method with Prior LiDAR Map Constrains

Han

Zou

Wang

et al. 2019

2019 IEEE International Conference on Robotics and Biomimetics (ROBIO)

Self Cite

View full text Add to dashboard Cite

In complex environments, low-cost and robust localization is a challenging problem. For example, in a GPSdenied environment, LiDAR can provide accurate position information, but the cost is high. In general, visual SLAM based localization methods become unreliable when the sunlight changes greatly. Therefore, inexpensive and reliable methods are required. In this paper, we propose a stereo visual localization method based on the prior LiDAR map. Different from the conventional visual localization system, we design a novel visual optimization model by matching planar information between the LiDAR map and visual image. Bundle adjustment is built by using coplanarity constraints. To solve the optimization problem, we use a graph-based optimization algorithm and a local window optimization method. Finally, we estimate a full six degrees of freedom (DOF) pose without scale drift. To validate the efficiency, the proposed method has been tested on the KITTI dataset. The results show that our method is more robust and accurate than the state-of-art ORB-SLAM2.

show abstract

“…These drawbacks result in long training time for the robot and limited generalization performance. Cloud robotic system [4] can be adopted to increase the learning efficiency of robots, and federated imitation learning algorithm is proposed to fuse the shared knowledge of robots. liuboyi17@mails.ucas.edu.cn; lj.wang1@siat.ac.cn 2 Ming liu is with Department of ECE, Hong Kong University of Science and Technology.…”

Section: Introductionmentioning

confidence: 99%

“…eelium@ust.hk 3 Boyi liu is also with the University of Chinese Academy of Sciences. 4 Cheng-Zhong Xu is with the University of Macau. czxu@um.edu.mo Fig.…”

Section: Introductionmentioning

confidence: 99%

Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems With Heterogeneous Sensor Data

Liu

Wang

Liu

et al. 2020

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Humans are capable of learning a new behavior by observing others to perform the skill. Similarly, robots can also implement this by imitation learning. Furthermore, if with external guidance, humans can master the new behavior more efficiently. So, how can robots achieve this? To address the issue, we present a novel framework named FIL. It provides a heterogeneous knowledge fusion mechanism for cloud robotic systems. Then, a knowledge fusion algorithm in FIL is proposed. It enables the cloud to fuse heterogeneous knowledge from local robots and generate guide models for robots with service requests. After that, we introduce a knowledge transfer scheme to facilitate local robots acquiring knowledge from the cloud. With FIL, a robot is capable of utilizing knowledge from other robots to increase its imitation learning in accuracy and efficiency. Compared with transfer learning and meta-learning, FIL is more suitable to be deployed in cloud robotic systems. Finally, we conduct experiments of a self-driving task for robots (cars). The experimental results demonstrate that the shared model generated by FIL increases imitation learning efficiency of local robots in cloud robotic systems.

show abstract

Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

Cited by 164 publications

References 19 publications

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

A Robust Stereo Camera Localization Method with Prior LiDAR Map Constrains

Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems With Heterogeneous Sensor Data

Contact Info

Product

Resources

About