A Closed-Form Solution to Non-rigid Shape and Motion Recovery

Abstract

Abstract. Recovery of three dimensional (3D) shape and motion of non-static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction. If every point in the scene randomly moves, it is impossible to recover the non-rigid shapes. In practice, many non-rigid objects, e.g. the human face under various expressions, deform with certain structures. Their shapes can be regarded as a weighted combination of certain shape bases. Shape and motion recovery under such situations has attracted much interest. Previous work on this problem [6,4,13] utilized only orthonormality constraints on the camera rotations (rotation constraints). This paper proves that using only the rotation constraints results in ambiguous and invalid solutions. The ambiguity arises from the fact that the shape bases are not unique because their linear transformation is a new set of eligible bases. To eliminate the ambiguity, we propose a set of novel constraints, basis constraints, which uniquely determine the shape bases. We prove that, under the weak-perspective projection model, enforcing both the basis and the rotation constraints leads to a closed-form solution to the problem of non-rigid shape and motion recovery. The accuracy and robustness of our closed-form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences.

Model-free non-rigid head pose tracking by joint shape and pose estimation

“…Building upon the rigid factorisation algorithm in [32], the proposed batch-processing method employed SVD to decompose a measurement matrix of higher rank into pose, basis shapes and their corresponding configuration weights. The formulation of the shape model proposed in [10] was adopted significantly by subsequent batch-processing [8,33,40,41] and sequential [1,5,11,27] approaches for non-rigid shape and motion recovery. A shortcoming of this model, however, is its sensitivity to the number of basis shapes that define the degrees-of-freedom of the surface deformations, where a restrictive model with too few basis shapes fails to model the measurement data well, while too many basis shapes in the model erroneously capture the noise in the data.…”

“…If the bases are not unique, an arbitrary linear transformation of a set of basis shapes produces a new set of eligible bases, leading to an ambiguous and invalid solution [40]. To avoid this ambiguity, different approaches imposed constraints on the problem of shape and motion recovery by introducing Gaussian priors on the configuration weights [5,33], coarse-to-fine ordering constraints on the deformation bases [8] and uniqueness constraints on the basis shapes [40]. To satisfy the latter constraint, for instance, the methods of [1,5] proposed to compute a set of meaningful basis shapes from a physics-based model of the surface of interest.…”

“…Methods which operate under the assumption of non-degeneracy introduce additional degreesof-freedom that, in the presence of degenerate deformations, are usually not descriptive of the measurement data and would erroneously capture the noise [1,5,8,10,11,33,40]. …”

Multi-View AAM Fitting and Construction

“…4.) ture-from-motion algorithms have be proposed, both nonlinear (Brand 2001;Torresani et al 2001) and linear (Bregler et al 2000;Xiao et al 2004b;Xiao and Kanade 2005) that can be used for deformable 3D model construction from both a single view (Brand 2001;Bregler et al 2000;Xiao et al 2004b;Xiao and Kanade 2005) and multiple views (Torresani et al 2001).…”

“…Two algorithms that takes this approach are (Cootes et al 1996;Gokturk et al 2001), one in the uncalibrated case (Cootes et al 1996), the other in the calibrated case (Gokturk et al 2001). An alternative approach is to extend the non-rigid structurefrom-motion paradigm of (Bregler et al 2000;Brand 2001;Torresani et al 2001;Xiao et al 2004b) and pose the face model construction problem as a single large optimization over the unknown shape model modes, in essence a large bundle adjustment. In Sect.…”