Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation

Tai, Lei; Paolo, Giuseppe; Liu, Ming

doi:10.1109/iros.2017.8202134

Cited by 680 publications

(484 citation statements)

References 8 publications

Supporting

Mentioning

475

Contrasting

Unclassified

Order By: Relevance

“…In robotics, simulations can be employed as an additional source of data [50], [51]. Physics simulators have been extensively developed for fields such as computer graphics or video gaming and one could potentially generate a vast amount of data.…”

Section: Motivating Examplesmentioning

confidence: 99%

A Review of Domain Adaptation without Target Labels

Kouw

Loog

2021

IEEE Trans. Pattern Anal. Mach. Intell.

377

213

View full text Add to dashboard Cite

Domain adaptation has become a prominent problem setting in machine learning and related fields. This review asks the question: how can a classifier learn from a source domain and generalize to a target domain? We present a categorization of approaches, divided into, what we refer to as, sample-based, feature-based and inference-based methods. Sample-based methods focus on weighting individual observations during training based on their importance to the target domain. Feature-based methods revolve around on mapping, projecting and representing features such that a source classifier performs well on the target domain and inference-based methods incorporate adaptation into the parameter estimation procedure, for instance through constraints on the optimization procedure. Additionally, we review a number of conditions that allow for formulating bounds on the cross-domain generalization error. Our categorization highlights recurring ideas and raises questions important to further research.

show abstract

Section: Motivating Examplesmentioning

confidence: 99%

A Review of Domain Adaptation without Target Labels

Kouw

Loog

2021

IEEE Trans. Pattern Anal. Mach. Intell.

377

213

View full text Add to dashboard Cite

show abstract

“…A. Reinforcement learning in robotics with simulations This work was motivated by the popularity of using reinforcement learning in robotics, despite RL being known to require large number of training samples and thus making it difficult to apply to robotics [9]. Part of this RL plus robotics work focuses on training policies in simulations and then transferring them to real robot, with or without further training on the robot [1].…”

Section: Arxiv:190500741v1 [Cslg] 2 May 2019mentioning

confidence: 99%

From Video Game to Real Robot: The Transfer Between Action Spaces

Karttunen

Kanervisto

Kyrki

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Training agents with reinforcement learning based techniques requires thousands of steps, which translates to long training periods when applied to robots. By training the policy in a simulated environment we avoid such limitation. Typically, the action spaces in a simulation and real robot are kept as similar as possible, but if we want to use a generic simulation environment, this strategy will not work. Video games, such as Doom (1993), offer a crude but multi-purpose environments that can used for learning various tasks. However, original Doom has four discrete actions for movement and the robot in our case has two continuous actions. In this work, we study the transfer between these two different action spaces. We begin with experiments in a simulated environment, after which we validate the results with experiments on a real robot. Results show that fine-tuning initially learned network parameters leads to unreliable results, but by keeping most of the neural network frozen we obtain above 90% success rate in simulation and real robot experiments.

show abstract

“…The "confidence value" motioned above of the actor is the degree of confirmation on which action the robot chooses to perform. For example, in a piece of sample from training data, the private Network 1 evaluates Q-values of different actions to (85, 85, 84, 83, 86), but the evaluation of the k-G sharing network is (20,20,100,10,10). In this case, we are more confident on actor of k-G sharing network, because it has significant differentiation in the scoring process.…”

Section: B Knowledge Fusion Algorithm In Cloudmentioning

confidence: 99%

Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

Liu

Wang

Liu

2019

IEEE Robot. Autom. Lett.

Self Cite

163

View full text Add to dashboard Cite

This paper was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the work, we propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website to provide the service based on LFRL: www.shared-robotics.com.

show abstract

Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation

Cited by 680 publications

References 8 publications

A Review of Domain Adaptation without Target Labels

A Review of Domain Adaptation without Target Labels

From Video Game to Real Robot: The Transfer Between Action Spaces

Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

Contact Info

Product

Resources

About