Learning Sequences of Manipulation Primitives for Robotic Assembly

Abstract

This paper explores the idea that skillful assembly is best represented as dynamic sequences of Manipulation Primitives, and that such sequences can be automatically discovered by Reinforcement Learning. Manipulation Primitives, such as "Move down until contact", "Slide along x while maintaining contact with the surface", have enough complexity to keep the search tree shallow, yet are generic enough to generalize across a wide range of assembly tasks. Policies are learned in simulation, and then transferred onto the physical platform. Direct sim2real transfer (without retraining in real) achieves excellent success rates on challenging assembly tasks, such as round peg insertion with 100 micron clearance or square peg insertion with large hole position/orientation estimation errors.

Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks

“…These values are sightly higher than the simulation and we hypothesize that the extra compliance offered by the impedance control helps the insertion process. Comparing with the continuous control baseline, two primitivebased methods (ours and discrete primitives) are more robust to sim-to-real transfer because using primitives as action improves the generalizability, which is also suggested in [11]. Moreover, benefiting from learning continuous primitive parameters, the primitives obtained by our proposed method are more efficient and have significantly higher success rates than using discrete primitives.…”

“…Similar to [11], we define one insertion primitive as the robot end-effector's desired motion, which consists of the desired movement command and a stopping condition as parameters. However, in their method, they proposed to use a set of discrete primitives parameters for each primitive and treat primitives as discrete actions.…”

“…Thus, the agent only has limited choices of primitives from a finite set, which may result in sub-optimal and less flexible policies if the set is too small. In [11], the authors designed 91 primitives in total to include enough primitives. Nevertheless, it is a large discrete action space to search and could make learning more challenging.…”

“…We consider three baseline comparisons with our proposed method, which are pure MP-DQN using parameterized action space, a continuous action baseline using continuous velocity control command as action, and a discrete primitive baseline proposed in [11]. The discrete primitives have six translation or rotation directions (±X, ±Y, ±Z), two velocity choices (v = 0.2 or 0.5), and four force thresholds (f lim = 1, 2, 3, +∞).…”

“…The concept of primitive learning has been applied in a wide range of applications, including bimanual manipulation [8], video games [9] and robotic soccer ball [10]. Vuong et al [11] introduced manipulation primitives to solve robotic assembly tasks and achieve good performance with real robots. However, their approach limits to discrete parameter selections for each primitive and is sensitive to hyperparameters such as the total number of parameter choices.…”