Learning to Touch Objects Through Stage-Wise Deep Reinforcement Learning

Bourdonnaye, Francois de La; Teulière, Céline; Triesch, Jochen; Chateau, Thierry

doi:10.1109/iros.2018.8593362

Cited by 7 publications

(2 citation statements)

References 20 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, their work have several limitations. Firstly, the task was executed with minimal consideration of human supervision in terms of kinematic models, calibration parameters or hand-crafted features [246]. Secondly, the detection usually requires processing large amounts of data, a process that is difficult and costly [43].…”

Section: Vision-based Robotic Graspmentioning

confidence: 99%

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

2020

View full text Add to dashboard Cite

The motivation behind our work is to review and analyze the most relevant studies on deep reinforcement learning-based object manipulation. Various studies are examined through a survey of existing literature and investigation of various aspects, namely, the intended applications, techniques applied, challenges faced by researchers and recommendations for minimizing obstacles. This review refers to all relevant articles on deep reinforcement learning-based object manipulation and solutions. The object grasping issue is a major manipulation challenge. Object grasping requires detection systems, methods and tools to facilitate efficient and fast agent training. Several studies have proposed that object grasping and its subtypes are the main elements in dealing with the environment and agent. Unlike other review articles, this review article provides different observations on deep reinforcement learning-based manipulation. The results of this comprehensive review of deep reinforcement learning in the manipulation field may be valuable for researchers and practitioners because they can expedite the establishment of important guidelines.

show abstract

Section: Vision-based Robotic Graspmentioning

confidence: 99%

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

2020

View full text Add to dashboard Cite

show abstract

“…The difficulty of our task relies on the fact that the arm has to reach from a large set of initial conditions (different object positions and initial arm positions, see Figure 5) so that it frequently has to substantially modify its orientation to reach the target with the palm. In this paper, we extend our prior work de La Bourdonnaye et al (2018) to the more complex setting of multiple object positions. Besides, we conduct additional experiments to study of the influence of different reward terms.…”

Section: Introductionmentioning

confidence: 99%

Stage-Wise Learning of Reaching Using Little Prior Knowledge

Bourdonnaye¹,

Teulière²,

Triesch³

et al. 2018

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

In some manipulation robotics environments, because of the difficulty of precisely modeling dynamics and computing features which describe well the variety of scene appearances, hand-programming a robot behavior is often intractable. Deep reinforcement learning methods partially alleviate this problem in that they can dispense with hand-crafted features for the state representation and do not need pre-computed dynamics. However, they often use prior information in the task definition in the form of shaping rewards which guide the robot toward goal state areas but require engineering or human supervision and can lead to sub-optimal behavior. In this work we consider a complex robot reaching task with a large range of initial object positions and initial arm positions and propose a new learning approach with minimal supervision. Inspired by developmental robotics, our method consists of a weakly-supervised stage-wise procedure of three tasks. First, the robot learns to fixate the object with a 2-camera system. Second, it learns hand-eye coordination by learning to fixate its end-effector. Third, using the knowledge acquired in the previous steps, it learns to reach the object at different positions and from a large set of initial robot joint angles. Experiments in a simulated environment show that our stage-wise framework yields similar reaching performances, compared with a supervised setting without using kinematic models, hand-crafted features, calibration parameters or supervised visual modules.

show abstract

Safe reward‐based deep reinforcement learning control for an electro‐hydraulic servo system

Liu

et al. 2022

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

In this article, a safe deep reinforcement learning (DRL) control method based on a safe reward shaping method is proposed and applied to the constrained control for an electro-hydraulic servo system (EHSS). The proposed control method improves the safety of the constrained control for a nonlinear system with the minimal intervention to the optimization of the performance objective, while the convergence speed of the DRL process has accelerated. By introducing control barrier functions (CBFs) to the reward shaping, a CBF-based potential difference term is designed to shape the safe reward, which not only provides the safe guidance for the DRL process by encoding the safety constraints of the nonlinear system, but also considers effects of the complex safety transformation on the convergence process in the DRL. Then the safe reward-based DRL control method is presented to learn the optimal safety policy of position tracking for the EHSS with position error constraints by planning and optimizing the safety together with the performance objective. Theoretical analysis is given to demonstrate that the proposed control method with the safe reward can achieve the optimal safety performance for the constrained control system.Experimental results of the constrained control for the EHSS with system uncertainties and perturbations are also exhibited, to show that the proposed control method converges fast and performs safer and better than the conventional control methods. K E Y W O R D Sconstrained control, control barrier function, deep reinforcement learning, electro-hydraulic servo system, safe reward shaping INTRODUCTIONDeep reinforcement learning (DRL) has been well used for nonlinear systems with uncertainties and complex dynamics, such as controlling robots 1-3 and self-driving vehicles, 4,5 because of its powerful ability to capture features from high-dimensional data and learn complicated control policies. 6 Several classical DRL algorithms including asynchronous advantage actor-critic (A3C), 7 deep Q-network, 8 and deep deterministic policy gradient (DDPG), 9 have been developedMinling Wu and Lijun Liu are co-first authors.

show abstract

Learning to Touch Objects Through Stage-Wise Deep Reinforcement Learning

Cited by 7 publications

References 20 publications

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

Stage-Wise Learning of Reaching Using Little Prior Knowledge

Safe reward‐based deep reinforcement learning control for an electro‐hydraulic servo system

Contact Info

Product

Resources

About