Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction

Reizenstein, Jeremy; Шаповалов, Роман; Henzler, Philipp; Sbordone, L.; Labatut, Patrick; Novotný, David

doi:10.1109/iccv48922.2021.01072

Cited by 182 publications

(106 citation statements)

References 38 publications

Supporting

Mentioning

101

Contrasting

Order By: Relevance

“…To obtain the rough initial geometry estimate D c , we use [51] at a reduced resolution of 64 × 64. Co3D [37] is a collection of nearly 19,000 videos capturing objects from 50 MS-COCO [26] categories, that come with per-frame depth, camera pose data, and reconstructed sparse point clouds. First, we use the Point Cloud Library [40] to compute surface normals from the point clouds.…”

Section: Datasets and Metricsmentioning

confidence: 99%

See 1 more Smart Citation

De-rendering 3D Objects in the Wild

Wimbauer¹,

Wu²,

Rupprecht³

2022

Preprint

View full text Add to dashboard Cite

De-renderingAlbedo Normals Diffuse Specular Diffuse Specular Rendering Rendering Figure 1. De-rendering 3D objects from in-the-wild images. Left: Our method is trained on unconstrained images with coarse shape estimates which can be obtained using off-the-shelf methods or classical structure from motion. Right: Our method de-renders an image into precise shape, material (diffuse albedo, specularity, specular intensity), and lighting (direction, intensity). The image can then be rerendered and re-lit based on our illumination model. The model generalizes to objects beyond the training categories (e.g. parkingmeter).

show abstract

Section: Datasets and Metricsmentioning

confidence: 99%

“…This estimate can come from a variety of sources. For datasets such as Co3D [37], where multi-view information is available, we rely on traditional structure-from-motion pipelines (e.g. COLMAP [42]).…”

Section: Introductionmentioning

confidence: 99%

De-rendering 3D Objects in the Wild

Wimbauer¹,

Wu²,

Rupprecht³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The densities for each point are predicted by aggregating information from all other points using a single attention layer. Similarly for category-specific reconstruction, NerFormer [38] proposed to replace the MLP in NeRF-WCE [23] with a transformer model to allow for spatial reasoning. Our work crucially differs from these methods at their core: the rendering framework.…”

Section: Related Workmentioning

confidence: 99%

Light Field Neural Rendering

Suhail¹,

Esteves²,

Sigal³

et al. 2021

Preprint

View full text Add to dashboard Cite

Classical light field rendering for novel view synthesis can accurately reproduce view-dependent effects such as reflection, refraction, and translucency, but requires a dense view sampling of the scene. Methods based on geometric reconstruction need only sparse views, but cannot accurately model non-Lambertian effects. We introduce a model that combines the strengths and mitigates the limitations of these two directions. By operating on a four-dimensional representation of the light field, our model learns to represent view-dependent effects accurately. By enforcing geometric constraints during training and inference, the scene geometry is implicitly learned from a sparse set of views. Concretely, we introduce a two-stage transformer-based model that first aggregates features along epipolar lines, then aggregates features along reference views to produce the color of a target ray. Our model outperforms the state-ofthe-art on multiple forward-facing and 360 • datasets, with larger margins on scenes with severe view-dependent variations. Code and results can be found at light-field-neuralrendering.github.io.* Work done while interning at Google. Ground TruthOurs (35.3 dB) NeX (30.4 dB) NeRF (29.6 dB)

show abstract

“…that operating systems now natively support viewing and editing 3D content (e.g., iOS/MacOS and Windows). In addition to curated 3D object datasets for research [5,14,20,44,58], large repositories of 3D shapes provide both synthetic [47,53,54] and scanned objects [22].…”

Section: Introductionmentioning

confidence: 99%