Tensor-Based Non-Rigid Structure from Motion

Grashof, Stella; Brandt, Sami S.

doi:10.1109/wacv51458.2022.00231

Cited by 9 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That said, our method uses an orthographic camera model with a low-rank shape assumption in NRSf M. Hence, by construction, it has some limitations for e.g., our method may perform inadequately on high perspective distortion images having large object deformation. A recent idea by Graßhof et al [21] can be used to overcome such a limitation. Finally, we conclude that the clever use of organic priors with matrix factorization theory is sufficient to provide excellent 3D reconstruction accuracy for both sparse and dense NRSf M.…”

Section: Discussionmentioning

confidence: 99%

Organic Priors in Non-Rigid Structure from Motion

Kumar¹,

Gool²

2022

Preprint

View full text Add to dashboard Cite

This paper advocates the use of organic priors in classical non-rigid structure from motion (NRSf M). By organic priors, we mean invaluable intermediate prior information intrinsic to the NRSf M matrix factorization theory. It is shown that such priors reside in the factorized matrices, and quite surprisingly, existing methods generally disregard them. The paper's main contribution is to put forward a simple, methodical, and practical method that can effectively exploit such organic priors to solve NRSf M. The proposed method does not make assumptions other than the popular one on the low-rank shape and offers a reliable solution to NRSf M under orthographic projection. Our work reveals that the accessibility of organic priors is independent of the camera motion and shape deformation type. Besides that, the paper provides insights into the NRSf M factorization-both in terms of shape and motion-and is the first approach to show the benefit of single rotation averaging for NRSf M. Furthermore, we outline how to effectively recover motion and non-rigid 3D shape using the proposed organic prior based approach and demonstrate results that outperform prior-free NRSf M performance by a significant margin. Finally, we present the benefits of our method via extensive experiments and evaluations on several benchmark datasets.

show abstract

Section: Discussionmentioning

confidence: 99%

Organic Priors in Non-Rigid Structure from Motion

Kumar¹,

Gool²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…NRSfM uses only weak prior assumptions about the observed motions and deformations and no 3D priors. Significant progress was achieved in comprehending and solving this classic ill-posed 3D computer vision problem over the last decades [Bra05, THB08, Graßhof and Brandt combine tensor-based modeling and rankone 3D shape basis formulation for NRSfM [GB22]. They recover 3D shapes up to an affine 3D transformation and perform a metric update if camera calibration is known.…”

Section: Non-rigid Structure From Motion (Nrsfm)mentioning

confidence: 99%

“…This allows to focus on the 3D reconstruction while delegating dense point tracking. At the same time-even if a method can accurately reconstruct a scene from accurate point tracks-it is often not known how the same approach performs on real and deteriorated 2D tracks [GRA13a,AGS17,GB22,WLPL22]. (Only several works evaluate the proposed methods on noise-contaminated ground-truth measurements [KCDL18, Kum19, PSF20, GJST20].)…”

Section: Non-rigid Structure From Motion (Nrsfm)mentioning

confidence: 99%

State of the Art in Dense Monocular Non‐Rigid 3D Reconstruction

Tretschk

Navami

Mallikarjun

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

3D reconstruction of deformable (or non‐rigid) scenes from a set of monocular 2D image observations is a long‐standing and actively researched area of computer vision and graphics. It is an ill‐posed inverse problem, since—without additional prior assumptions—it permits infinitely many solutions leading to accurate projection to the input 2D images. Non‐rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set‐ups such as stereo or multi‐view systems. This survey focuses on state‐of‐the‐art methods for dense non‐rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods—that handle arbitrary scenes and make only a few prior assumptions—and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high‐level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.

show abstract

“…However, because the ideal number of bases is typically unpredictable (different bases occur in various sequences), this strategy is not robust. Recently, NRSfM has advanced significantly as the BMM algorithm [20], isometric priors [21], elastic priors [22], local-rigidity prior [23,24], space-time smooth constraints [25], variational methods [7], and tensor-based models [26] have been successively proposed.…”

Section: Related Workmentioning

confidence: 99%

Deep Spatial-Temporal Neural Network for Dense Non-Rigid Structure from Motion

Wang

Huang

et al. 2022

Mathematics

View full text Add to dashboard Cite

Dense non-rigid structure from motion (NRSfM) has long been a challenge in computer vision because of the vast number of feature points. As neural networks develop rapidly, a novel solution is emerging. However, existing methods ignore the significance of spatial–temporal data and the strong capacity of neural networks for learning. This study proposes a deep spatial–temporal NRSfM framework (DST-NRSfM) and introduces a weighted spatial constraint to further optimize the 3D reconstruction results. Layer normalization layers are applied in dense NRSfM tasks to stop gradient disappearance and hasten neural network convergence. Our DST-NRSfM framework outperforms both classical approaches and recent advancements. It achieves state-of-the-art performance across commonly used synthetic and real benchmark datasets.

show abstract

Tensor-Based Non-Rigid Structure from Motion

Cited by 9 publications

References 42 publications

Organic Priors in Non-Rigid Structure from Motion

Organic Priors in Non-Rigid Structure from Motion

State of the Art in Dense Monocular Non‐Rigid 3D Reconstruction

Deep Spatial-Temporal Neural Network for Dense Non-Rigid Structure from Motion

Contact Info

Product

Resources

About