Incremental Learning of Skills in a Task-Parameterized Gaussian Mixture Model

Abstract

Programming by demonstration techniques facilitate the programming of robots. Some of them allow the generalization of tasks through parameters, although they require new training when trajectories different from the ones used to estimate the model need to be added. One of the ways to re-train a robot is by incremental learning, which supplies additional information of the task and does not require teaching the whole task again. The present study proposes three techniques to add trajectories to a previously estimated task-parameterized Gaussian mixture model. The first technique estimates a new model by accumulating the new trajectory and the set of trajectories generated using the previous model. The second technique permits adding to the parameters of the existent model those obtained for the new trajectories. The third one updates the model parameters by running a modified version of the Expectation-Maximization algorithm, with the information of the new trajectories. The techniques were evaluated in a simulated task and a real one, and they showed better performance than that of the existent model.

Quantifying teaching behavior in robot learning from demonstration

“…Feedback from robot learners to human teachers in LfD is often considered in pragmatic terms, with solutions determined based on the task at hand, or simply assuming the teacher will be able to successfully interpret the learner’s actions and adjust their teaching behavior accordingly. Incremental methods in LfD are viewed as a way of gradually learning a skill, and can be adapted to help improve and/or overcome a users physical skill deficiencies (Calinon and Billard, 2007a; Hoyos et al, 2016; Tykal et al, 2016). However, even though the process is iterative, there has been little research on how novice users interpret and adapt to the learner over consecutive teaching steps.…”

Hierarchical Task-Parameterized Learning from Demonstration for Collaborative Object Movement

“…Note that here we stored all demonstrations and computed a new TP-GMM every time for VT, instead of using one of the incremental TP-GMM methods in [23] because of the following reasons. First, the generative technique in [23] does not save computation time compared to VT because it samples trajectories using the existing model to represent previously encoded trajectories, which are then encoded with new trajectories to form a completely new model. Furthermore, performance may suffer because sampled trajectories are used in the new model instead of actual demonstrations.…”

“…Furthermore, performance may suffer because sampled trajectories are used in the new model instead of actual demonstrations. Second, the model addition technique in [23] will also take strictly more time than HS because it encodes new demonstrations in a new model like HS and then has to concatenate and optimize the previous and the new models together. Third, the direct update technique [23] assumes that the old demonstrations and the new ones are drawn from the same distribution, which is problematic because we sample from a relatively large number of task situations or even an outlier situation.…”

“…In contrary, there is no inherent method in VT to identify the outlier, and even if a human operator identifies the outlier situation to be deleted, a new TP-GMM has to be trained from scratch to recover the performance of the nine remaining inlier demonstrations. Incremental methods such as [15, 23] do not seem to consider removing demonstrations.…”

“…Other examples that incrementally build skill libraries from new demonstrations include [20–22]. Existing methods that incrementally train GMMs have also been adapted for TP-GMMs [23].…”