Deformation capture and modeling of soft objects

Abstract

We present a data-driven method for deformation capture and modeling of general soft objects. We adopt an iterative framework that consists of one component for physics-based deformation tracking and another for spacetime optimization of deformation parameters. Low cost depth sensors are used for the deformation capture, and we do not require any force-displacement measurements, thus making the data capture a cheap and convenient process. We augment a state-of-the-art probabilistic tracking method to robustly handle noise, occlusions, fast movements and large deformations. The spacetime optimization aims to match the simulated trajectories with the tracked ones. The optimized deformation model is then used to boost the accuracy of the tracking results, which can in turn improve the deformation parameter estimation itself in later iterations. Numerical experiments demonstrate that the tracking and parameter optimization components complement each other nicely. Our spacetime optimization of the deformation model includes not only the material elasticity parameters and dynamic damping coefficients, but also the reference shape which can differ significantly from the static shape for soft objects. The resulting optimization problem is highly nonlinear in high dimensions, and challenging to solve with previous methods. We propose a novel splitting algorithm that alternates between reference shape optimization and deformation parameter estimation, and thus enables tailoring the optimization of each subproblem more efficiently and robustly. Our system enables realistic motion reconstruction as well as synthesis of virtual soft objects in response to user stimulation. Validation experiments show that our method not only is accurate, but also compares favorably to existing techniques. We also showcase the ability of our system with high quality animations generated from optimized deformation parameters for a variety of soft objects, such as live plants and fabricated models.

Bayesian estimation of non-rigid mechanical parameters using temporal sequences of deformation samples

“…We also compare our method with the most recent work by Wang et al [30]. They also use temporal sequences of deformation samples as input, but they assume the deformable objects are in static state.…”

“…Lee et al proposed a model to estimate the Young's modulus based on low-resolution CT images and no external force is required to set the boundary condition [4]. The most recent work on identification of mechanical properties based on surface tracking [30] proposed a decoupled iterative tracking and parameter estimation framework. They applied a combined probabilistic physically-based method for surface tracking.…”

“…Comparison with results of[30] and experimental measurements.THE MEASURED YOUNG'S MODULUS FOR THE TENNIS BALL IS TAKEN FROM [40], [41] AND FOR THE FOAM BALL IS DERIVED FROM [42]. THE RECOVERED YOUNG'S MODULUS USING OUR METHOD IS WITHIN THE RANGE OF THE YOUNG'S MODULUS MEASURED, THE YOUNG'S MODULUS RECOVERED USING [30] IS NOT.…”

Cloth Manipulation Using Random-Forest-Based Imitation Learning

“…On the other hand, deformable body tracking solves a simpler problem, namely inferring the 3D configuration of a deformable object from sensing inputs. There is literature on deformable body tracking, which infers the 3D configuration from sensor data [13,14,15]. However, these methods usually require a template mesh as a priori and are mainly limited to handling small deformations.…”

Inverse elastic shell design with contact and friction

“…Furthermore, even in the absence of frictional contact, inverting a soft membrane subject to gravity is particularly challenging due to the large displacements that occur between the natural shape and its shape at equilibrium. We have tried in the past several inverse algorithms that were said to be successful in other contexts, for instance in the case of soft objects modeled with co-rotated linear FEM [Wang et al 2015], but they have all failed when applied to our problem. We have thus striven to design a particularly robust inversion algorithm, able to account not only for gravity applied onto a soft thin shell, but also for contact and dry friction with an external body.…”