Probabilistic simultaneous pose and non-rigid shape recovery

Moreno-Noguer, Francesc; Porta, Josep M.

doi:10.1109/cvpr.2011.5995532

Cited by 30 publications

(51 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a low-rank shape basis has been computed by learning techniques like principal component analysis over a set of training data [12,29], applying modal [1,9] or spectral [4] analysis over a rest configuration, or estimated on-line using data-driven methods [20,26,31,39].…”

Section: Classical Global Low-rank Shape Modelmentioning

confidence: 99%

Global Model with Local Interpretation for Dynamic Shape Reconstruction

Agudo

Moreno-Noguer

2017

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

Self Cite

View full text Add to dashboard Cite

The most standard approach to resolve the inherent ambiguities of the non-rigid structure from motion problem is using low-rank models that approximate deforming shapes by a linear combination of rigid basis. These models are typically global, i.e., each shape basis contributes equally to all points of the surface. While this approach has been shown effective to represent smooth deformations, its performance degrades for surfaces composed of various regions, each following a different deformation rule. Piecewise methods attempt to capture this type of behavior by locally modeling surface patches, although they subsequently require enforcing global constraints to assemble back the patches. In this paper we propose an approach that combines the best of global and local models: it locally considers low-rank models but, by construction, does not need to impose global constraints to guarantee local patch continuity. We achieve this by a simple expectation maximization strategy that besides learning global shape bases, it locally adapts their contribution to each specific surface region. Furthermore, as a side contribution, in order to split the surface into different local patches, we propose a novel physically-based mesh segmentation approach that obeys an energy criterion. The complete framework is evaluated in both synthetic and real datasets, and shows an improved performance to competing methods.

show abstract

Section: Classical Global Low-rank Shape Modelmentioning

confidence: 99%

Global Model with Local Interpretation for Dynamic Shape Reconstruction

Agudo

Moreno-Noguer

2017

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In (Moreno-Noguer and Porta 2011;Salzmann et al 2008) the unknown shape was encoded as a linear combination of deformation modes learnt in advance from a relatively large set of training data. Ostlund et al 2012 introduced the Laplacian formalism in computer vision to regularize 3D meshes without requiring any training data.…”

Section: Related Workmentioning

confidence: 99%

Modal Space: A Physics-Based Model for Sequential Estimation of Time-Varying Shape from Monocular Video

2016

View full text Add to dashboard Cite

This paper describes two sequential methods for recovering the camera pose together with the 3D shape of highly deformable surfaces from a monocular video. The non-rigid 3D shape is modeled as a linear combination of mode shapes with time-varying weights that define the shape at each frame and are estimated on-the-fly. The low-rank constraint is combined with standard smoothness priors to optimize the model parameters over a sliding window of image frames. We propose to obtain a physics-based shape basis using the initial frames on the video to code the timevarying shape along the sequence, reducing the problem from trilinear to bilinear. To this end, the 3D shape is discretized by means of a soup of elastic triangular finite elements where we apply a force balance equation. This equation is solved using modal analysis via a simple eigenvalue problem to obtain a shape basis that encodes the modes of deformation. Even though this strategy can be applied in a wide variety of scenarios, when the observations are denser, the solution can become prohibitive in terms of computational load. We avoid this limitation by proposing two efficient coarse-tofine approaches that allow us to easily deal with dense 3D surfaces. This results in a scalable solution that estimates a small number of parameters per frame and could potentially run in real time. We show results on both synthetic and real videos with ground truth 3D data, while robustly dealing with artifacts such as noise and missing data.

show abstract

“…This work is an extended version of our earlier paper [27] where we already proposed the probabilistic framework to integrate parameters describing both the camera motion and the surface deformation. Here, we exploit the generality of this methodology and show that it allows introducing additional constraints.…”

Section: Introductionmentioning

confidence: 99%

A Bayesian approach to simultaneously recover camera pose and non-rigid shape from monocular images

Moreno-Noguer

Porta

2016

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

In this paper we bring the tools of the Simultaneous Localization and Map Building (SLAM) problem from a rigid to a deformable domain and use them to simultaneously recover the 3D shape of non-rigid surfaces and the sequence of poses of a moving camera. Under the assumption that the surface shape may be represented as a weighted sum of deformation modes, we show that the problem of estimating the modal weights along with the camera poses, can be probabilistically formulated as a maximum a posteriori estimate and solved using an iterative least squares optimization. In addition, the probabilistic formulation we propose is very general and allows introducing different constraints without requiring any extra complexity. As a proof of concept, we show that local inextensibility constraints that prevent the surface from stretching can be easily integrated.An extensive evaluation on synthetic and real data, demonstrates that our method has several advantages over current non-rigid shape from motion approaches. In particular, we show that our solution is robust to large amounts of noise and outliers and that it does not need to track points over the whole sequence nor to use an initialization close from the ground truth.

show abstract

Probabilistic simultaneous pose and non-rigid shape recovery

Cited by 30 publications

References 26 publications

Global Model with Local Interpretation for Dynamic Shape Reconstruction

Global Model with Local Interpretation for Dynamic Shape Reconstruction

Modal Space: A Physics-Based Model for Sequential Estimation of Time-Varying Shape from Monocular Video

A Bayesian approach to simultaneously recover camera pose and non-rigid shape from monocular images

Contact Info

Product

Resources

About