Learning motor skills with non-rigid materials by reinforcement learning

Abstract

This paper focuses on learning motor skills for anthropomorphic robots which must interact with non-rigid materials to perform tasks, such as wearing clothes, turning socks inside out, and bandaging. To learn such a motor skill, the task to be performed needs to be quantitatively defined using not only the state of the robot, but also the state of the non-rigid material. However, the non-rigid material is generally represented in a high dimensional configuration space (e.g., [1]) and obtaining such information in a real environment is difficult. In this paper we propose a novel learning framework for learning motor skills interacting with non-rigid materials by reinforcement learning that avoids these difficulties. Our learning framework focuses on the topological relationship between the configuration of the robot and the non-rigid material based on the consideration that most details of the material (e.g., wrinkles) are not important for performing the motor tasks. This focus allows us to define the task performance and provide reward signals based on a low-dimensional variable and to measure task performance in a real environment using reliable sensors. We constructed an experimental setting with an anthropomorphic dual-arm robot and a tailor-made T-shirt for the robot. To demonstrate the feasibility of the proposed method, we applied the method to have the robot perform the motor task of putting on the T-shirt. As a result of our learning framework, through trial and error the robot was able to acquire sequential movements that performed the goal of putting both arms into the corresponding sleeves of the T-shirt.

Reinforcement learning of clothing assistance with a dual-arm robot

“…A study of Shinohara et al [1] shares similar research aspects with our study. They considered that the relationship between the configuration of the robot and the nonrigid material is very important to achieve motor tasks, and have proposed to use the topological coordinates [5] for the state and reward representation of the cloth.…”

“…Namely we need to have low-dimensional representations for the cloth as well as the arm controller. This study employs the learning scheme that Shinohara et al have proposed [1]. Although the most popular approach to detect and grasp non-rigid materials is to use image sensors, e.g., [2], [3], it is not suitable for this study, since the non-rigid materials are generally represented in a high dimensional space, e.g., [4], which prevents the completion of reinforcement learning within reasonable time.…”

Execution Fault Recovery in Robot Programming by Demonstration Using Multiple Models

“…Algunos son [8] y [9]. En [8] se presenta una técnica en la que dos brazos robóticos, colocan entre sus propios brazos una camiseta.…”

“…Con esta información y mediante aprendizaje por refuerzo el robot logra colocar la camiseta. Nuestro trabajo se diferencia de [8] y [9], en que éstos usan aprendizaje por refuerzo y además requieren un modelo en coordenadas topológicas de las mangas y hueco de la camiseta. También, con respecto a [9], el maniquí es fijo, a diferencia de nuestra propuesta que admite variaciones en posición y curvatura del brazo del maniquí.…”

“…Algunos son [8] y [9]. En [8] se presenta una técnica en la que dos brazos robóticos, colocan entre sus propios brazos una camiseta. Los autores usan una técnica que combina aprendizaje por refuerzo, visión estéreo, y coordenadas topológicas, para obtener un modelo de la tarea compuesto por el estado del robot y el de la camiseta.…”