DeepDeform: Learning Non-Rigid RGB-D Reconstruction With Semi-Supervised Data

Božič, Aljaž; Zollhöfer, Michael; Theobalt, Christian; NieBner, Matthias

doi:10.1109/cvpr42600.2020.00703

Cited by 52 publications

(27 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to unambiguously and reliably represent and describe these objects, we need to compute invariant features of the object's under these non-rigid deformations. Therefore, non-rigid tracking and reconstruction (Bozic et al, 2020) are tasks that can directly profit from methods that robustly handle correspondences of surfaces under deformation. Some existing keypoint description approaches approximate geometric intrinsic cues solely from image intensity, such as done in DaLI descriptor (Moreno-Noguer, 2011), however they suffer from high computational costs and loss of distinctiveness in exchange for its robustness to deformations.…”

Section: Arxiv:220312016v1 [Cscv] 22 Mar 2022mentioning

confidence: 99%

Learning geodesic-aware local features from RGB-D images

Potje

Martins²,

Cadar³

et al. 2022

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Section: Arxiv:220312016v1 [Cscv] 22 Mar 2022mentioning

confidence: 99%

Learning geodesic-aware local features from RGB-D images

Potje

Martins²,

Cadar³

et al. 2022

Computer Vision and Image Understanding

View full text Add to dashboard Cite

“…Volumetric fusion based methods [32,56,59,61,66] allow free-form dynamic reconstruction in a template-free, single-view, real-time way, through updating depth into the canonical model and performing non-rigid deformation. A series of works are proposed to make volumetric fusion more robust with SIFT features [17], human articulated skeleton prior [59,61], extra IMU sensors [66], data-driven prior [43], learned correspondences [3] or neural deformation graph [2]. Since these single-view setups suffer from tracking error in the occluded parts, multi-view setups are introduced to mitigate this problem with improved fusion methods.…”

Section: Related Workmentioning

confidence: 99%

NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions

Jiang¹,

Jiang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

4D modeling of human-object interactions is critical for numerous applications. However, efficient volumetric capture and rendering of complex interaction scenarios, especially from sparse inputs, remain challenging. In this paper, we propose NeuralFusion, a neural approach for volumetric human-object capture and rendering using sparse consumer RGBD sensors. It marries traditional non-rigid fusion with recent neural implicit modeling and blending advances, where the captured humans and objects are layerwise disentangled. For geometry modeling, we propose a neural implicit inference scheme with non-rigid key-volume fusion, as well as a template-aid robust object tracking pipeline. Our scheme enables detailed and complete geometry generation under complex interactions and occlusions. Moreover, we introduce a layer-wise human-object texture rendering scheme, which combines volumetric and image-based rendering in both spatial and temporal domains to obtain photo-realistic results. Extensive experiments demonstrate the effectiveness and efficiency of our approach in synthesizing photo-realistic free-view results under complex human-object interactions.

show abstract

“…Pirors like skeleton [Yu et al 2017], parametric body shape or inertial measurement units ] are used to facilitate the fusion. [Bozic et al 2020] applies data-driven approaches for non-rigid 3D reconstruction. Rather than using a strict photometric consistency criterion, [Lombardi et al 2019] learn a generative model that tries to best match the input images without assuming that objects in the scene are compositions of flat surfaces.…”

Section: Related Workmentioning

confidence: 99%