Towards latent space based manipulation of elastic rods using autoencoder models and robust centerline extractions

Abstract

The automatic shape control of deformable objects is an open (and currently hot) manipulation problem that is challenging due to the object's high-dimensional shape information and its complex physical properties. As a feasible solution to these issues, in this paper, we propose a new methodology to automatically deform elastic rods into 2D desired shapes. For that, we present an efficient method that uses the Deep Autoencoder Network (DAE) to compute a compact representation of the object's infinite dimensional shape. To deal with the (typically unknown) properties, we use an online algorithm that approximates the sensorimotor mapping between the robot's configuration and the object's shape features. Our new approach has the capability to compute the rod's centerline from raw visual data and in real-time. This is done by introducing an innovative algorithm based on the self-organizing network (SOM). The effectiveness of the proposed method is thoroughly validated with numerical simulations and experiments.

Model Predictive Manipulation of Compliant Objects with Multi-Objective Optimizer and Adversarial Network for Occlusion Compensation

“…Other geometric features computed from contours and centerlines [9]- [11] can represent soft object deformation in a more general way. Various data-driven approaches have also been proposed to represent shapes, e.g., using fast point feature histograms [12], bottleneck layers [13], [14], principal component analysis [15], etc. However, there is no widely accepted approach to compute efficient/compact feature representations for 3D shape; This is still an open research problem.…”

Particle Filters in Latent Space for Robust Deformable Linear Object Tracking

“…Yan et al [36] use the latent dynamics models from offline learning to solve deformable object manipulation tasks such as spreading ropes and cloths. Similarly, Qi et al [37] use an autoencoder to learn a compact representation of DLOs shape which is used in a vision-based controller for manipulation. Ma et al [38] propose latent graph dynamics for deformable object manipulation which abstracts the deformable object state as a low-dimensional keypointbased graph with learned latent features.…”

Motion-planning based Robot Manipulation

“…The unique properties of this class of objects must be taken into consideration, necessitating particular approaches to them [4], [5]. Due to the difficulty of computing all conceivable interactions, systems designed to handle the manipulation of such objects face problems with processing times [6]- [8].…”