Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction

Abstract

We present a general paradigm for dynamic 3D reconstruction from multiple independent and uncontrolled image sources having arbitrary temporal sampling density and distribution. Our graph-theoretic formulation models the spatio-temporal relationships among our observations in terms of the joint estimation of their 3D geometry and its discrete Laplace operator. Towards this end, we define a tri-convex optimization framework that leverages the geometric properties and dependencies found among a Euclidean shape-space and the discrete Laplace operator describing its local and global topology. We present a reconstructability analysis, experiments on motion capture data and multi-view image datasets, as well as explore applications to geometrybased event segmentation and data association.

Unsupervised 3D Reconstruction and Grouping of Rigid and Non-Rigid Categories

“…The most standard prior used in NRSfM consists in constraining the deforming shape to lie in a low-rank subspace. In order to learn such a low-rank model, early approaches rely on factorization [13], [31], [48], [57], or optimization-based strategies [12], [39], [55], [58]. More recently, the low-rank constraint has been enforced by means of PCA-like formulations in which…”

“…Early works addressed this problem under the assumption of a rigid structure [1], [41], [46]. Later, many efforts were focused on the non-rigid case, to retrieve dynamic 3D shape and camera motion from only 2D measurements in a monocular video [5], [33], [39], [52], [55], [58]. This problem is known to be inherently ambiguous and demanded introducing more sophisticated priors.…”

Segmentation and 3D Reconstruction of NON-RIGID Shape from RGB Video

“…Retrieving a non-rigid 3D structure together with the camera motion from solely the observation of 2D point tracks in a monocular video is an ill-posed problem that requires to exploit the art of priors. The most used idea to address the problem is to assume that the 3D shape lies in a low-rank subspace defined by shape [5,11], trajectory [6,7], shapetrajectory [12,13,14], or force [8] vectors. The main limitation of previous approaches is the dimensionality of the subspace is known in advance, making them very problem specific.…”

“…(11) .102 .147(7) .102 .148(7) .088 .115(10) .106 .158(11) .106 .143 0.3(2) .091 .133 0.2(2) Pick-up .417 .423(14) .249 .429(5) .155 .237(12) .155 .230(6) .155 .233(6) .121 .173(12) .154 .235(12) .154 .221 3.7(3) .…”

Piecewise Bézier Space: Recovering 3D Dynamic Motion From Video

“…Unfortunately, solving this problem without 3D supervision is an ill-posed problem that requires to explore the art of priors, being them more sophisticated than those utilized in the rigid case. Maybe, the most popular priors are based on low-rank subspaces constraining the solution space of either the entire shape [1,2,3,4], the 3D point trajectories [5,6], shape-trajectory combinations [7,8,9,10] or the force patterns that induce the deformations [11]. Similar shape [12] and trajectory [13] subspaces have also been exploited in a deep learning context, where large amounts of training data are needed to learn the model.…”