Active perception and modeling of deformable surfaces using Gaussian processes and position-based dynamics

Abstract

Exploring and modeling heterogeneous elastic surfaces requires multiple interactions with the environment and a complex selection of physical material parameters. The most common approaches model deformable properties from sets of offline observations using computationally expensive forcebased simulators. In this work we present an online probabilistic framework for autonomous estimation of a deformability distribution map of heterogeneous elastic surfaces from few physical interactions. The method takes advantage of Gaussian Processes for constructing a model of the environment geometry surrounding a robot. A fast Position-based Dynamics simulator uses focused environmental observations in order to model the elastic behavior of portions of the environment. Gaussian Process Regression maps the local deformability on the whole environment in order to generate a deformability distribution map. We show experimental results using a PrimeSense camera, a Kinova Jaco2 robotic arm and an Optoforce sensor on different deformable surfaces.

Multi-Modal Sensing and Robotic Manipulation of Non-Rigid Objects: A Survey

“…In this approach, however, visual data from the Kinect sensor are used to detect and track the deformation while the force measurement is used as a complement in the case that a grasp adjustment is needed. Caccamo et al [29] proposed a system for modeling deformation of elastic surfaces. The setup consisted of a PrimeSense RGB-D camera a Kinova Jaco24 robotic arm equipped with a 3 fingered Kinova KG-3 gripper that carried a 3D OptoForce force sensor.…”

“…Fugl et al [9] Frank et al [25] Leizea et al [10] Petit et al [26] Lin et al [11] Güler et al [5] Caccamo et al [29] Hui et al [6] Cretu et al [22] Arriola-Rios and Wyatt [21] Drimus et al [14] Mira et al [16] Delgado et al [18] Navarro-Alarcon et al [12] Alambeigi et al [13]…”

“…In principle, the model is able to simulate several levels of deformation, however for this work only shear-and stretch-like deformations are considered, which is controlled by a Process in a simpler, generic robotic experimental setup, parameter named β. Afterwards, Caccamo et al [29] included a Gaussian Proccess Regression (GPR) to the PBD methodology in order to estimate the deformation of a elastic heterogeneous surface. Gil et al 's [28] grasping strategy used colour and depth data and geometric information to be able to model the shape deformation of a planar object.…”

Combining Self-Organizing and Graph Neural Networks for Modeling Deformable Objects in Robotic Manipulation

“…All the sensors and hardware components used in the robotic manipulation setup are operated through ROS (Quigley et al, 2009). The data preparation, signal and image processing steps are implemented using SciPy (Virtanen et al, 2020) and OpenCV (Bradski, 2000) libraries. The models are implemented in the Deep Graph Library (Wang et al, 2019) using PyTorch (Paszke et al, 2019) as backend.…”

Data_Sheet_1.PDF

“…Other methods combine analytical and data-driven approaches in different parts of the modeling pipeline. For example, a Gaussian Process Regression (GPR) is used to estimate the deformability parameter of a PBD model (Caccamo et al, 2016). An Evolutionary Algorithm (EA) is proposed to search for the parameter space of an MSS model (Arriola-Rios and Wyatt, 2017).…”