A Motion Capture and Imitation Learning Based Approach to Robot Control

Abstract

Imitation learning is a discipline of machine learning primarily concerned with replicating observed behavior of agents known to perform well on a given task, collected in demonstration data sets. In this paper, we set out to introduce a pipeline for collecting demonstrations and training models that can produce motion plans for industrial robots. Object throwing is defined as the motivating use case. Multiple input data modalities are surveyed, and motion capture is selected as the most practicable. Two model architectures operating autoregressively are examined—feedforward and recurrent neural networks. Trained models execute throws on a real robot successfully, and a battery of quantitative evaluation metrics is proposed. Recurrent neural networks outperform feedforward ones in most respects, but this advantage is not universal or conclusive. The data collection, pre-processing and model training aspects of our proposed approach show promise, but further work is required in developing Cartesian motion planning tools before it is applicable in production applications.

Design and Programming of a Robotic Puppetry Robot Based on Natural Learner Unit Pattern Generators Neural Networks

“…In this method, the teacher participates in the robot's learning process and moves its joints according to its movement constraints. This direct mapping does not change the content of the display data and is only a reference frame in which the data are displayed [31].…”

Tracking humano visual aplicado a manipuladores no antropomórficos para imitación

“…Dentro de las demostraciones indirectas, y en tareas de manipulación, un enfoque típico es replicar movimientos humanos. Para el seguimiento preciso de personas, a menudo se utilizan sistemas de captura de movimiento (MoCap), formados por dispositivos de visión complejos (Racinskis et al, 2022). Estos tienen claras desventajas, como la necesidad de grandes espacios para usar el sistema o de marcadores corporales como referencia, lo que impide su aplicación directa en entornos domésticos.…”