Hierarchical structure and nonrigid motion recovery from 2D monocular views

Abstract

Inferring both 3D structure andmotion of nonrigid obje cts from monocular images is an important problem in computational vision. The challenges stem not only fr om the absenc e o f p oint corresp ondences but also from the structur eambiguity. In this paper, a hierarchical method which integrates both local patch analysis and global shap e descriptions is devise d to solve the dual problem of structur e andnonrigid motion recovery by using an elastic geometric model { Extended Sup er quadrics (ESQ). The nonrigid object of interest is segmented into many small areas and local analysis is performed t o r ecover small details for each small area, assuming that each small area is undergoing similar nonrigid motion. Then, a recursive algorithm is propose dto guide and regularize local analysis with glob al information by using an appropriate global ESQ model. This local-global hierarchy enables us to captur eboth local and global deformations accurately and robustly. Experimental results on both simulation and real data are p r esente d to validate and evaluate the e e ctiveness and robustness of the prop osed a p p r oach.

Extending Superquadrics with Exponent Functions: Modeling and Reconstruction

“…In [31], extended superquadrics were used to construct a human face model which can model details of human faces and is proven to be very useful for both 3D head motion recovery and visualization. In [33], extended superquadrics were used as global motion models in order to track both structure and nonrigid motion of elastic objects such as human faces. In our future work, physical properties will be incorporated into the extended superquadrics in order to model and visualize real-world dynamics and behaviors.…”

Nonrigid motion recovery for 3D surfaces

“…(31) at each point of interest for x, y and z dimensions, respectively) and each row of A is the right part of Eqs. (31), (33) or (37). B is a (4nC6)!3(mC4) dimensional matrix and each row of B is the left part of Eqs.…”

“…B is a (4nC6)!3(mC4) dimensional matrix and each row of B is the left part of Eqs. (31), (33) or (37) without PKP 0 .…”

“…It has many applications such as in medical image registration [4,12,28,30], face modeling [29,33] and remote sensing [34,35]. Comprehensive reviews of early nonrigid motion analysis methods can be found in Refs.…”

Active blobs: region-based, deformable appearance models

“…One of the problems of fitting a 3D model to a single image is that depth is almost impossible to determine, and the tracking problem is generally underconstrained. As a result, 3D model-based nonrigid motion tracking techniques require either strong prior assumptions about the shape of the object (e.g., symmetry [57], deformable superquadrics [37,40,66], human faces [15,17,56]) or that multiple views of the object are available. By using a 2D deformable image template, the active blobs formulation has the advantage of being able to track general nonrigid motion within the image plane without the need to define a full 3D model nor strong prior assumptions about the shape of the object to be tracked.…”