Sparse Dynamic 3D Reconstruction from Unsynchronized Videos

Zheng, Enliang; Ji, Dinghuang; Dunn, Enrique; Frahm, Jan‐Michael

doi:10.1109/iccv.2015.504

Cited by 24 publications

(16 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Along these lines, Avidan and Shashua [6] estimate dynamic geometry from 2D observations of points constrained to linear and conical motions. However, under the assumption of dense temporal motion sampling, the concept of motion smoothness has been successfully exploited [25,26,45,46,35,42,43,36,30,31]. Park et al [25] triangulate 3D point trajectories by the linear combination of Direct Cosine Transform trajectory bases with the constraint of a reprojection system.…”

Section: Related Workmentioning

confidence: 99%

“…Valmadre et al [35] reduce the number of trajectory bases by setting a gain threshold depending on the basis null-space and propose a method using a highpass filter to mitigate low reconstructability for scenarios having no missing 2D observations. Zheng et al [43,42] propose a dictionary learning method to estimate the 3D shape with partial sequencing info, assuming 3D geometry estimates may be approximated by local barycentric interpolation (i.e. self-expressive motion prior) and developed a bi-convex framework for jointly estimating 3D geometry and barycentric weights.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction

Dunn

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

We present a general paradigm for dynamic 3D reconstruction from multiple independent and uncontrolled image sources having arbitrary temporal sampling density and distribution. Our graph-theoretic formulation models the spatio-temporal relationships among our observations in terms of the joint estimation of their 3D geometry and its discrete Laplace operator. Towards this end, we define a tri-convex optimization framework that leverages the geometric properties and dependencies found among a Euclidean shape-space and the discrete Laplace operator describing its local and global topology. We present a reconstructability analysis, experiments on motion capture data and multi-view image datasets, as well as explore applications to geometrybased event segmentation and data association.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction

Dunn

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…For conventional single view depth sequences and multiple view reconstruction of dynamic scenes techniques have been introduced to align sequences using correspondence information between frames. Methods have been proposed to obtain sparse [22,23,24,25] and dense [26,27,28] correspondence between consecutive frames for entire sequences. Existing sparse correspondence methods work independently on a frame-by-frame basis for a single view [25] or multiple views [23] and require a strong prior initialization [24].…”

Section: D Reconstruction Of Dynamic Scenesmentioning

confidence: 99%

4D Temporally Coherent Light-Field Video

Mustafa

Volino

Guillemaut

et al. 2017

2017 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

Light-field video has recently been used in virtual and augmented reality applications to increase realism and immersion. However, existing light-field methods are generally limited to static scenes due to the requirement to acquire a dense scene representation. The large amount of data and the absence of methods to infer temporal coherence pose major challenges in storage, compression and editing compared to conventional video. In this paper, we propose the first method to extract a spatio-temporally coherent light-field video representation. A novel method to obtain Epipolar Plane Images (EPIs) from a spare lightfield camera array is proposed. EPIs are used to constrain scene flow estimation to obtain 4D temporally coherent representations of dynamic light-fields. Temporal coherence is achieved on a variety of light-field datasets. Evaluation of the proposed light-field scene flow against existing multiview dense correspondence approaches demonstrates a significant improvement in accuracy of temporal coherence.

show abstract

“…Methods have been proposed to obtain sparse [16,17,14] and dense [15,18,13] correspondence between consecutive frames for entire sequence. Existing sparse correspondence methods work sequentially on a frame-to-frame basis for single view [14] or multi-view [16] and require a strong prior initialization [17]. Existing dense matching or scene flow methods [12,13] require a strong prior which fails in the case of large motion and moving cameras.…”

Section: Related Workmentioning

confidence: 99%

“…Scene flow techniques [12,13] typically estimate the pairwise surface or volume correspondence between reconstructions at successive frames but do not extend to 4D alignment or correspondence across complete sequences due to drift and failure for rapid and complex motion. Existing feature matching techniques either work in 2D [14] or 3D [15] or for sparse [16,17] or dense [18] points. However these methods fail in the case of occlusion, large motions, background clutter, deformation, moving cameras and appearance of new parts of objects.…”

Section: Introductionmentioning

confidence: 99%

4D Match Trees for Non-rigid Surface Alignment

Mustafa

Kim

Hilton

2016

Computer Vision – ECCV 2016

View full text Add to dashboard Cite

Abstract. This paper presents a method for dense 4D temporal alignment of partial reconstructions of non-rigid surfaces observed from single or multiple moving cameras of complex scenes. 4D Match Trees are introduced for robust global alignment of non-rigid shape based on the similarity between images across sequences and views. Wide-timeframe sparse correspondence between arbitrary pairs of images is established using a segmentation-based feature detector (SFD) which is demonstrated to give improved matching of non-rigid shape. Sparse SFD correspondence allows the similarity between any pair of image frames to be estimated for moving cameras and multiple views. This enables the 4D Match Tree to be constructed which minimises the observed change in non-rigid shape for global alignment across all images. Dense 4D temporal correspondence across all frames is then estimated by traversing the 4D Match tree using optical flow initialised from the sparse feature matches. The approach is evaluated on single and multiple view images sequences for alignment of partial surface reconstructions of dynamic objects in complex indoor and outdoor scenes to obtain a temporally consistent 4D representation. Comparison to previous 2D and 3D scene flow demonstrates that 4D Match Trees achieve reduced errors due to drift and improved robustness to large non-rigid deformations.

show abstract

Sparse Dynamic 3D Reconstruction from Unsynchronized Videos

Cited by 24 publications

References 29 publications

Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction

Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction

4D Temporally Coherent Light-Field Video

4D Match Trees for Non-rigid Surface Alignment

Contact Info

Product

Resources

About