Nonrigid Surface Registration and Completion from RGBD Images

Abstract

Abstract. Nonrigid surface registration is a challenging problem that suffers from many ambiguities. Existing methods typically assume the availability of full volumetric data, or require a global model of the surface of interest. In this paper, we introduce an approach to nonrigid registration that performs on relatively low-quality RGBD images and does not assume prior knowledge of the global surface shape. To this end, we model the surface as a collection of patches, and infer the patch deformations by performing inference in a graphical model. Our representation lets us fill in the holes in the input depth maps, thus essentially achieving surface completion. Our experimental evaluation demonstrates the effectiveness of our approach on several sequences, as well as its robustness to missing data and occlusions.

Deformable 3D Fusion: From Partial Dynamic 3D Observations to Complete 4D Models

“…Kinect sequence depicting a deforming sheet of paper 2 , and, as in [37], augmented the data with a synthetic occluder, which hides roughly half of the surface in the first frame, and is then progressively removed. The results in the bottom row of Fig.…”

“…In particular, some techniques tackle the case of open surfaces, where the surface can be entirely observed in at least some frames of the sequence [37,16]. While some work has tackled the more challenging case of volumetric surfaces, most existing methods rely on a pre-processing step, where a 3D model of the object of interest is acquired under rigid, or quasi-rigid motion [17,7,18,35,39,40,42].…”

A Survey of Non‐Rigid 3D Registration

“…[XSWL14] also perform registration with a patch‐based deformation. Instead of rigid transformation, each patch is deformed by a displacement field using a linear combination of deformation modes learned from simulated data.…”

NRST: Non-rigid Surface Tracking from Monocular Video

“…Although multi-view methods can produce accurate tracking results, their setup is expensive and hard to operate. Some approaches use a single RGB-D sensor instead [42,44,19,18,9,10,36]. They manage to capture deformable surfaces nicely and at high efficiency, some even build up a template model alongside per-frame reconstruction.…”

“…Most of the existing methods are based on multi-view imagery, where expensive and complicated system setups are required [3,25,23]. There also exist methods that rely on only a single depth or RGB-D camera [42,44,19,18]. However, these sensors are not as ubiquitous as RGB cameras, and these methods cannot be applied on plenty of existing video Fig.…”