Knowledge-Driven Deep Deterministic Policy Gradient for Robotic Multiple Peg-in-Hole Assembly Tasks

Hou, Zhimin; Dong, Haiming; Zhang, Kuangen; Gao, Quan; Chen, Ken; Xu, Jing

doi:10.1109/robio.2018.8665255

Cited by 17 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A deep Q-learning network was used to model the learning process of assembly skills [15]. Hou et al proposed the knowledge-driven deep deterministic policy gradient algorithm for robotic multiple peg-in-hole assembly tasks [34]. The present work deals with the peg-hole-based assembly of rigid parts.…”

Section: B Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Manipulation Skill Acquisition for Robotic Assembly Based on Multi-Modal Information Description

Li¹,

Jiang²,

Quan³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Automatic assembly of elastic components is difficult because of the potential deformation of parts during the assembly process. Consequently, robots cannot adapt their manipulation to dynamic changes. Designing systems that learn assembly skills can help in alleviating the uncertain factor for industrialgrade assembly robots. This study proposes a skill acquisition method based on multi-modal information description to realize the assembly of systems with elastic components. This multi-modal information includes two-dimensional images, poses, forces/torques, and robot joint parameters. In this method, robots acquire searching, location determination, and pose adjustment skills using these multi-modal information parameters. As a result, robots can reach the assembly target by analyzing two-dimensional images with no position constraint. While acquiring pose adjustment skills, the reward function with depth and assembly steps is used to improve the learning efficiency. The deep deterministic policy gradient (DDPG) algorithm is applied for acquiring skills. Experiments using a KUKA iiwa robot demonstrated the effectiveness and conciseness of our method. Our results indicate that the robot acquired searching, location determination, and pose adjustment skills that allowed it to successfully complete elastic assembly. INDEX TERMS Industrial robots, acquisition of manipulation skills, deep reinforcement learning, multi-modal information description. I. INTRODUCTION A. MOTIVATION Robots are used for more efficient production assembly. Research on automation of assembly systems has focused on intelligent, automatic, and flexible robots. During an actual assembly process, force measurement data are not available owing to elastic deformation between workpieces. Elastic deformation can be neglected when dealing with assembly of rigid parts. However, in the case of weakly rigid components, such deformation causes a series of uncertain changes, such as position, posture, and depth changes. There are methods based on force-control and compliance control that are already facing the problem of assembly of elastic The associate editor coordinating the review of this article and approving it for publication was Okyay Kaynak.

show abstract

Section: B Related Workmentioning

confidence: 99%

“…The assembly components might be damaged owing to the imprecise pose. Compared with the peg-hole task [33], [34], the object complexity increases the difficulty of robotic skills acquisition. Given the above, the main problem to be solved is to make robots acquire skills of object location determination and pose adjustment.…”

Section: B Problem Setupmentioning

confidence: 99%

Manipulation Skill Acquisition for Robotic Assembly Based on Multi-Modal Information Description

Li¹,

Jiang²,

Quan³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…An actor-critic method was also used to complete high-precision assembly tasks (Wang et al, 2019). However, these methods need to discretize the action space to output discrete actions (Hou et al, 2018;Beltran-Hernandez et al, 2020a). The most significant limitation of this approach is the curse of dimensionality: as the degrees of freedom increase, the number of actions will increase exponentially (Lillicrap et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

“…For tasks that require precise control of motion, the situation is even worse because they require finer discretization. This fine-grained discretization will lead to ineffective exploration in a large action space and reduce the learning speed (Hou et al, 2018;Lillicrap et al, 2015). Therefore, these methods cannot well solve the problems in continuous and high-dimensional action space, such as robot control (Lillicrap et al, 2015;Beltran-Hernandez et al, 2020a).…”

Section: Introductionmentioning

confidence: 99%

“…Although this method has achieved good performance, it does not consider improving the DDPG algorithm itself to further improve the learning efficiency of the RL agent. Hou et al (2018) proposed a knowledge-driven DDPG algorithm to complete the task of multiple peg-in-hole assembly. However, the knowledge base greatly affects the learning effect of the agent.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multiple peg-in-hole compliant assembly based on a learning-accelerated deep deterministic policy gradient strategy

Xiao

Zhao

et al. 2021

View full text Add to dashboard Cite

Purpose As complex analysis of contact models is required in the traditional assembly strategy, it is still a challenge for a robot to complete the multiple peg-in-hole assembly tasks autonomously. This paper aims to enable the robot to complete the assembly tasks autonomously and more efficiently, with the strategies learned by reinforcement learning (RL), a learning-accelerated deep deterministic policy gradient (LADDPG) algorithm is proposed. Design/methodology/approach The multiple peg-in-hole assembly strategy is designed in two modules: an advanced planning module and a bottom control module. The advanced module is completed by the LADDPG agent, which is used to derive advanced commands based on geometric and environmental constraints, that is, the desired contact force. The bottom-level control module will drive the robot to complete the compliant assembly task through the adaptive impedance algorithm according to the command set issued by the advanced module. In addition, a set of safety assurance mechanisms is developed to safely train a collaborative robot to complete autonomous learning. Findings The method can complete the assembly tasks well through RL, and it can realize satisfactory compliance of the robot to the environment. Compared with the original DDPG algorithm, the average values of the instantaneous maximum contact force and contact torque during the assembly process are reduced by approximately 38% and 74%, respectively. Practical implications The entire algorithm can also be applied to other robots and the assembly strategy can be applied in the field of the automatic assembly. Originality/value A compliant assembly strategy based on the LADDPG algorithm is proposed to complete the automated multiple peg-in-hole assembly tasks.

show abstract

Reinforcement Learning With Stereo-View Observation for Robust Electronic Component Robotic Insertion

Bartyzel,

Półchłopek,

Rzepka

2023

J Intell Robot Syst

View full text Add to dashboard Cite

In modern manufacturing, assembly tasks are a major challenge for robotics. In the manufacturing industry, a wide range of insertion tasks can be found, from peg-in-hole insertion to electronic parts assembly. Robotic stations designed for this problem often use conventional hybrid force-position control to perform preprogrammed trajectories, such as e.g. a spiral path. However, electronic parts require more sophisticated techniques due to their complex geometry and susceptibility to damage. Production line assembly tasks require high robustness to initial position and rotation variations due to component grip imperfections. Robustness to partially obscured camera view is also mandatory due to multi stage assembly process. We propose a stereo-view method based on reinforcement learning (RL) for the robust assembly of electronic parts. Applicability of our method to real-world production lines is verified through test scenarios. Our approach is the most robust to applied perturbations of all tested methods and can potentially be transferred to environments unseen during learning.

show abstract

Knowledge-Driven Deep Deterministic Policy Gradient for Robotic Multiple Peg-in-Hole Assembly Tasks

Cited by 17 publications

References 7 publications

Manipulation Skill Acquisition for Robotic Assembly Based on Multi-Modal Information Description

Manipulation Skill Acquisition for Robotic Assembly Based on Multi-Modal Information Description

Multiple peg-in-hole compliant assembly based on a learning-accelerated deep deterministic policy gradient strategy

Reinforcement Learning With Stereo-View Observation for Robust Electronic Component Robotic Insertion

Contact Info

Product

Resources

About