A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations

Abstract

In manufacturing, traditional task pre-programming methods limit the efficiency of human–robot skill transfer. This paper proposes a novel task-learning strategy, enabling robots to learn skills from human demonstrations flexibly and generalize skills under new task situations. Specifically, we establish a markerless vision capture system to acquire continuous human hand movements and develop a threshold-based heuristic segmentation algorithm to segment the complete movements into different movement primitives (MPs) which encode human hand movements with task-oriented models. For movement primitive learning, we adopt a Gaussian mixture model and Gaussian mixture regression (GMM-GMR) to extract the optimal trajectory encapsulating sufficient human features and utilize dynamical movement primitives (DMPs) to learn for trajectory generalization. In addition, we propose an improved visuo-spatial skill learning (VSL) algorithm to learn goal configurations concerning spatial relationships between task-relevant objects. Only one multioperation demonstration is required for learning, and robots can generalize goal configurations under new task situations following the task execution order from demonstration. A series of peg-in-hole experiments demonstrate that the proposed task-learning strategy can obtain exact pick-and-place points and generate smooth human-like trajectories, verifying the effectiveness of the proposed strategy.

Experimental Evaluation of the Success of Peg-in-Hole Tasks Learned from Demonstration

“…The advantages of PbD over the traditional programming can be listed as follows [1] [2] [3] [4] [5]:…”

“…The first issue is the development and implementation of overarching frameworks that address the learning of the whole assembly process from start to finish [4] [6]. Such frameworks often rely on vision-based systems to decompose a demonstrated assembly operation into a sequence of smaller tasks.…”

“…These tasks are then learned using methods such as Gaussian Mixture Models (GMM), Hidden Markov Models (HMM), or Dynamic Movement Primitives (DMP). The proposed frameworks, on the other hand, represent the relationship between the individual tasks and the overall process, so that an appropriate sequence of tasks can be generated when encountered with a novel assembly operation [4] [7].…”

“…For example, a peg-in-hole task is generalized if the peg can still be inserted into the hole successfully, even if the hole is at a different position, or its dimensions are slightly different. To accomplish this, learning methods such as DMP can generate new trajectories for novel hole and peg poses, using the data recorded during the demonstrations [4]. Even though the methods such as DMP can provide some level of generalization capability, there is no guarantee that all the generalizations of a task will be successful.…”

“…Even though the methods such as DMP can provide some level of generalization capability, there is no guarantee that all the generalizations of a task will be successful. For example, the literature suggests that the success rate for a peg-in-hole task can be as high as 90% [4], or as low as 50% [8]. Trivially, the demonstrations themselves can be replayed with a practically 100% success rate.…”