A Survey of Sim-to-Real Transfer Techniques Applied to Reinforcement Learning for Bioinspired Robots

Zhu, Wei; Guo, Xian; Owaki, Dai; Kutsuzawa, Kyo; Hayashibe, Mitsuhiro

doi:10.1109/tnnls.2021.3112718

Cited by 16 publications

(10 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, most of real applications utilize wheeled mobile robots with stable motion properties [2], [3], [24]. Some simto-real applications are implemented on humanoid robots with omni-directional motion ability [29], [30]. However, it is rare to deploy the DRL-based navigation policy on wheel bipedal robots with unstable motion characteristics [7].…”

Section: Related Workmentioning

confidence: 99%

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

Zhu

Hayashibe

2023

IEEE Trans. Ind. Electron.

Self Cite

View full text Add to dashboard Cite

We present a hierarchical deep reinforcement learning (DRL) framework with prominent sampling efficiency and sim-to-real transfer ability for fast and safe navigation: the low-level DRL policy enables the robot to move towards the target position and keep a safe distance to obstacles simultaneously; the high-level DRL policy is supplemented to further enhance the navigation safety. We select a waypoint located on the path from the robot to the ultimate goal as the sub-goal to reduce the state space and avoid sparse reward. Moreover, the path is generated based on either a local or a global map, which can significantly improve the sampling efficiency, safety, and generalization ability of the proposed DRL framework. Additionally, a target-directed representation for the action space can be derived based on the sub-goal to improve the motion efficiency and reduce the action space. In order to demonstrate the eminent sampling efficiency, motion performance, obstacle avoidance, and generalization ability of the proposed framework, we implement sufficient comparisons with the non-learning navigation methods and DRL-based baselines, with videos, data, code, and other supplemental material shown on our website 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

Zhu

Hayashibe

2023

IEEE Trans. Ind. Electron.

Self Cite

View full text Add to dashboard Cite

show abstract

“…So for the learner, the real world is just a disturbed environment. More details of the sim-to-real problem can be seen in Zhu et al ( 2021 ).…”

Section: Dexterous Manipulation For Multi-fingered Robotic Hands With...mentioning

confidence: 99%

Dexterous Manipulation for Multi-Fingered Robotic Hands With Reinforcement Learning: A Review

Wang

2022

Front. Neurorobot.

View full text Add to dashboard Cite

With the increasing demand for the dexterity of robotic operation, dexterous manipulation of multi-fingered robotic hands with reinforcement learning is an interesting subject in the field of robotics research. Our purpose is to present a comprehensive review of the techniques for dexterous manipulation with multi-fingered robotic hands, such as the model-based approach without learning in early years, and the latest research and methodologies focused on the method based on reinforcement learning and its variations. This work attempts to summarize the evolution and the state of the art in this field and provide a summary of the current challenges and future directions in a way that allows future researchers to understand this field.

show abstract

“…Silver et al [29] reached a new milestone in deep RL with AlphaGo, an agent trained to play Go that became the first computer program capable of beating a professional human Go player. Even with its immense progress, most deep RL applications and test environments are still limited to games and simulations [30]. Recently, to improve RL's applicability to the real world, more researchers have recognized the importance of learning from static datasets of observations.…”

Section: Introductionmentioning

confidence: 99%

A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems

Prudencio

Máximo

Colombini

2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

With the widespread adoption of deep learning, reinforcement learning (RL) has experienced a dramatic increase in popularity, scaling to previously intractable problems, such as playing complex games from pixel observations, sustaining conversations with humans, and controlling robotic agents. However, there is still a wide range of domains inaccessible to RL due to the high cost and danger of interacting with the environment. Offline RL is a paradigm that learns exclusively from static datasets of previously collected interactions, making it feasible to extract policies from large and diverse training datasets. Effective offline RL algorithms have a much wider range of applications than online RL, being particularly appealing for real-world applications, such as education, healthcare, and robotics. In this work, we contribute with a unifying taxonomy to classify offline RL methods. Furthermore, we provide a comprehensive review of the latest algorithmic breakthroughs in the field using a unified notation as well as a review of existing benchmarks' properties and shortcomings. Additionally, we provide a figure that summarizes the performance of each method and class of methods on different dataset properties, equipping researchers with the tools to decide which type of algorithm is best suited for the problem at hand and identify which classes of algorithms look the most promising. Finally, we provide our perspective on open problems and propose future research directions for this rapidly growing field.

show abstract

A Survey of Sim-to-Real Transfer Techniques Applied to Reinforcement Learning for Bioinspired Robots

Cited by 16 publications

References 79 publications

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

Dexterous Manipulation for Multi-Fingered Robotic Hands With Reinforcement Learning: A Review

A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems

Contact Info

Product

Resources

About