Domain randomization for transferring deep neural networks from simulation to the real world

Tobin, Josh; Fong, Rachel; Ray, Alex; Schneider, Jonas; Zaremba, Wojciech; Abbeel, Pieter

doi:10.1109/iros.2017.8202133

Cited by 2,006 publications

(1,579 citation statements)

References 55 publications

Supporting

Mentioning

1,572

Contrasting

Unclassified

Order By: Relevance

“…This makes directly transferring policies from the simulator to the real world difficult. However, Sadeghi and Levine [40] and other researchers [47,49] show how randomizing textures and lighting allows for effective transfer to the real world. We employ these domain randomization techniques to transfer to a real robot.…”

Section: Imagementioning

confidence: 99%

“…By combining our Asymmetric Actor Critic training with domain randomization [47], we show that these visual policies can Pick, Push and Move a block using a real robot without any training on the physical system (see Figure 1).…”

Section: Imagementioning

confidence: 99%

“…A promising approach is domain adaptation by domain randomization [47,40]. Here the key idea is to train on randomized renderings of the scene, which supports learning robust policies for transfer.…”

Section: Transfer From Simulation To the Real Worldmentioning

confidence: 99%

“…A powerful technique for domain transfer of policies from rendered images to real world images is domain randomization [47,40]. The key idea is to randomize visual elements in the scene during the rendering.…”

Section: Randomization For Transfermentioning

confidence: 99%

“…One common method for robot learning using simulators is by training a visual policy on rendered images and then transferring it to a real robot [40,39,47]. These visual policies allow for easy instrumentation on the robot's end.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Asymmetric Actor Critic for Image-Based Robot Learning

Pinto¹,

Andrychowicz²,

Welinder³

et al. 2018

Robotics: Science and Systems XIV

Self Cite

157

149

View full text Add to dashboard Cite

Abstract-Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains. However, robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator. While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator. In this work, we propose the Asymmetric Actor Critic, which learns a vision-based control policy while taking advantage of access to the underlying state to significantly speed up training. Concretely, our algorithm employs an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) is trained on images. We show that using these asymmetric inputs improves performance on a range of simulated tasks. Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block. We achieve this simulation to real-world transfer without training on any real-world data. Videos of these experiments can be found in www.goo.gl/b57WTs.

show abstract

Section: Imagementioning

confidence: 99%

Section: Imagementioning

confidence: 99%

Section: Transfer From Simulation To the Real Worldmentioning

confidence: 99%

Section: Randomization For Transfermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Asymmetric Actor Critic for Image-Based Robot Learning

Pinto¹,

Andrychowicz²,

Welinder³

et al. 2018

Robotics: Science and Systems XIV

Self Cite

157

149

View full text Add to dashboard Cite

show abstract

Transfer learning for high‐precision trajectory tracking through adaptive feedback and iterative learning

Pereida

Kooijman

Duivenvoorden

et al. 2018

Adaptive Control & Signal

View full text Add to dashboard Cite

Robust and adaptive control strategies are needed when robots or automated systems are introduced to unknown and dynamic environments where they are required to cope with disturbances, unmodeled dynamics, and parametric uncertainties. In this paper, we demonstrate the capabilities of a combined  1 adaptive control and iterative learning control (ILC) framework to achieve high-precision trajectory tracking in the presence of unknown and changing disturbances. The  1 adaptive controller makes the system behave close to a reference model; however, it does not guarantee that perfect trajectory tracking is achieved, while ILC improves trajectory tracking performance based on previous iterations. The combined framework in this paper uses  1 adaptive control as an underlying controller that achieves a robust and repeatable behavior, while the ILC acts as a high-level adaptation scheme that mainly compensates for systematic tracking errors. We illustrate that this framework enables transfer learning between dynamically different systems, where learned experience of one system can be shown to be beneficial for another different system. Experimental results with two different quadrotors show the superior performance of the combined  1 -ILC framework compared with approaches using ILC with an underlying proportional-derivative controller or proportional-integral-derivative controller.Results highlight that our  1 -ILC framework can achieve high-precision trajectory tracking when unknown and changing disturbances are present and can achieve transfer of learned experience between dynamically different systems. Moreover, our approach is able to achieve precise trajectory tracking in the first attempt when the initial input is generated based on the reference model of the adaptive controller. KEYWORDSiterative learning,  1 adaptive control, trajectory tracking, transfer learning INTRODUCTIONRobots and automated systems are being deployed in unstructured and continuously changing environments. Sophisticated control methods are required to guarantee high overall performance in these environments where model uncertainties, unknown disturbances, and changing dynamics are present. Examples of robotic applications in unknown 388

show abstract

Dynamic Motion Planning Model for Multirobot Using Graph Neural Network and Historical Information

Qiu

2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

In order to effectively improve the path‐finding capability of a multirobot system in a decentralized control approach, a dynamic motion planning model based on graph neural networks and historical information (GNNHIM) is proposed. Due to the limited sensing range of the robot's onboard sensors, GNNHIM uses convolutional neural networks to extract features from the heterogeneous environmental information collected and analyzes the motion trends of other robots in conjunction with the historical path information stored locally. After finishing feature fusion, each robot exchanges its feature vectors with other robots still within sensing range to obtain the next action by the graph neural network model processing. Herein, imitation learning is used to train robots to choose behavioral strategies to maximize team benefit. The experimental results show that GNNHIM can make path planning for multirobot systems more efficient and reliable, enabling all robots to reach their own goal within a limited time in 95.1% of the experimental dataset. Moreover, GNNHIM still has great generalization when applied to unseen scenarios and larger scale robot systems. GNNHIM in unseen scenarios has improved the success rate by 6%‐34% and reduced the detour percentage by 3.8% on average compared to the other scheduling model.

show abstract

Domain randomization for transferring deep neural networks from simulation to the real world

Cited by 2,006 publications

References 55 publications

Asymmetric Actor Critic for Image-Based Robot Learning

Asymmetric Actor Critic for Image-Based Robot Learning

Transfer learning for high‐precision trajectory tracking through adaptive feedback and iterative learning

Dynamic Motion Planning Model for Multirobot Using Graph Neural Network and Historical Information

Contact Info

Product

Resources

About