Imitation Learning for Nonprehensile Manipulation Through Self-Supervised Learning Considering Motion Speed

Abstract

Robots are expected to replace menial tasks such as housework. Some of these tasks include nonprehensile manipulation performed without grasping objects. Nonprehensile manipulation is very difficult because it requires considering the dynamics of environments and objects. Imitating complex behaviors requires a large number of human demonstrations. In this study, a self-supervised learning that considers motion speed to achieve variable speed for nonprehensile manipulation is proposed. The proposed method collects and fine-tunes only successful actions among the data obtained during autonomous operations. By fine-tuning the successful data using speed labels, the robot learns the dynamics among itself, its environment, and objects. We experimented with the task of scooping and transporting pancakes using the neural network model trained on 24 human-collected training data. The proposed method significantly improved the success rate from 40.2% to 85.7%, and succeeded more than 75% for other objects.

Recalling Unknown Manipulations by Spontaneously Sharing Actions with Similar Objects in Observation Based Learning

“…起する手法を提案する．想起した複数の動作候補は，次の step で実行動作を決定するための選択肢として利用さ れる． いくつかの関連研究において，ロボットに物体操作方法を学習させ，未知のシーンに対して動作を想起させる アプローチが提案されている．ロボットに特定のシーンに対する人の動作を学習させて類似シーンで模倣した動 作を実行させる研究 (Saigusa et al, 2022) (Ruppel and Zhang, 2020)や物体画像に対する 3 次元ワイプ動作を学習して 類似物体形状の画像に対して動作を想起させる研究 (Saito et al, 2020)，シーンと物体に対する動作の関係を記憶 し，未知のシーンに対して最も近い学習シーンの動作を実行させる研究 (Rothfuss et al, 2018) (3)…”

Cooking Robot Mastery: Learning Online Motion Generation with Active Perception

“…To make a robot complete the "desired" daily dexterous manipulation tasks, learning humans' "skills" is necessary. To learn humans' skill, learning from demonstration (LfD) [29,30], which allows the robot to learn from expert examples without explicitly indicated commands, is critical, like other cooking robotics and dexterous manipulation research as [31,32,33,34]. We collected expert demonstration data with teleoperation and allowed the robot to acquire human cooking skills through LfD.…”