Simpler Learning of Robotic Manipulation of Clothing by Utilizing DIY Smart Textile Technology

Abstract

Deformable objects such as ropes, wires, and clothing are omnipresent in society and industry but are little researched in robotics research. This is due to the infinite amount of possible state configurations caused by the deformations of the deformable object. Engineered approaches try to cope with this by implementing highly complex operations in order to estimate the state of the deformable object. This complexity can be circumvented by utilizing learning-based approaches, such as reinforcement learning, which can deal with the intrinsic high-dimensional state space of deformable objects. However, the reward function in reinforcement learning needs to measure the state configuration of the highly deformable object. Vision-based reward functions are difficult to implement, given the high dimensionality of the state and complex dynamic behavior. In this work, we propose the consideration of concepts beyond vision and incorporate other modalities which can be extracted from deformable objects. By integrating tactile sensor cells into a textile piece, proprioceptive capabilities are gained that are valuable as they provide a reward function to a reinforcement learning agent. We demonstrate on a low-cost dual robotic arm setup that a physical agent can learn on a single CPU core to fold a rectangular patch of textile in the real world based on a learned reward function from tactile information.

Modular Piezoresistive Smart Textile for State Estimation of Cloths

“…In [ 19 ], an accessible, low-cost smart textile was developed to solve the problems associated with using vision for the state estimation of cloths. By integrating a flexible tactile sensor grid into a flat textile piece, a pose estimation model can be trained.…”

“…In other work [ 15 , 16 , 17 , 18 , 19 , 20 , 21 ], smart textiles have additionally been used for classification tasks in the field of robotics. In [ 18 ], flexible piezoresistive pressure sensors similar to the one in [ 6 ] are applied to a robotic gripper, enabling it to recognize different objects by grasping them.…”

“…In the same way that state estimation for soft robots is challenging, so is state estimation of cloths under robotic manipulation. In our previous work [ 19 ], we developed a piezoresistive smart textile that is able to classify its own deformation state and showed that an inexpensive dual robotic platform can learn to fold a textile piece in the real world with neural fitted Q-learning and without reward engineering. This approach provides an alternative to computationally expensive vision-based pipelines [ 22 , 23 ].…”

“…This approach provides an alternative to computationally expensive vision-based pipelines [ 22 , 23 ]. While [ 19 ] presented a valid proof-of-concept of smart textile state estimation for robotic learning of textile folding, the textile used was not suitable for application to larger cloths due to its limited flexibility and the footprint (size and weight) of the readout electronics. In order to reduce the weight and the size of the smart cloth, wired connections are often used to communicate with a signal processing unit separate from the sensor substrate (e.g., [ 16 , 17 , 18 , 20 , 21 ]).…”

Grasp Planning Pipeline for Robust Manipulation of 3D Deformable Objects with Industrial Robotic Hand + Arm Systems

“…An online Gaussian process regression model was used to estimate the deformation function of the manipulated objects and low-dimension features described the object's configuration. Finally, new works (such as [48,49]) integrate deep-learning techniques, 3D vision, and tactile information in order to fold/unfold and pick-and-place clothes. Although all these works consider the deformation of the object while manipulating it, an initial stable grasp is supposed to be known and nonlinear force-deformation relations are not computed during the handling process.…”