Canonical Surface Mapping via Geometric Cycle Consistency

Kulkarni, Nilesh; Tulsiani, Shubham; Gupta, Abhinav

doi:10.1109/iccv.2019.00229

Cited by 109 publications

(139 citation statements)

References 43 publications

(55 reference statements)

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…We report the average across ten randomly selected templates for each category. For a particular selected template, this baseline is an upper bound for CSM [5].…”

Section: Resultsmentioning

confidence: 99%

“…It generates impressive textured reconstructions from a single image, but still requires extra supervision in the form of corresponding keypoints during training, whereas we infer them. More recently, Canonical Surface Mapping (CSM) [5] predicts a UV mapping from a single image onto a canonical model, trained entirely using self-supervision, by introducing a geometric cycle-consistency term. For Kulkarni et al [10], the same mapping is applied but the canonical surface mesh can deform given an articulation parameter, which allows shape alignment to an input image.…”

Section: Related Workmentioning

confidence: 99%

“…Similar to CSM [5], we employ cycle consistency in our reconstruction loss to learn the parametric surface mapping: after mapping a pixel location p in an image to the surface mesh via the template, it should project back to the original location p. We further encourage that the projected point lies within a segmentation mask X v of the depicted shape, in each view v. The segmentation mask can be obtained with an off-the-shelf instance segmentation algorithm [19].…”

Section: Optimizationmentioning

confidence: 99%

“…The approach has two steps. First, as opposed to the Kulkarni et al [5] who use a fixed template, we generate an initial shape template with a multi-view shape encoder, which extends AtlasNet [2] to multiple viewpoints. Second, this template, specific to the shape we want to reconstruct, is refined using a cycle-consistency loss which allows the shape to adapt and finely align with its depiction in the input views.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

3D Reconstruction By Parameterized Surface Mapping

Langlois

Fisher

Wang

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

We introduce an approach for computing a 3D mesh from one or more views of an object by establishing dense correspondences between pixels in the views and 3D locations on a learnable parameterized surface. We propose a multi-view shape encoder that can be jointly trained with the AtlasNet surface parameterization. The shape is further refined using a novel geometric cycle-consistency loss between the learnable parameterized surface and input views. We demonstrate the efficacy of our approach on the ShapeNet-COCO dataset.

show abstract

“…We report the average across ten randomly selected templates for each category. For a particular selected template, this baseline is an upper bound for CSM [5].…”

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Optimizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

3D Reconstruction By Parameterized Surface Mapping

Langlois

Fisher

Wang

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…However, there is no control over the pose and style of the generated object. Alongside, in 3D computer vision, there are several attempts to learn object reconstruction [26,32,34,33],…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Object Detection via Generative Image Synthesis

Mustikovela¹,

Mello²,

Prakash³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present SSOD -the first end-to-end analysis-bysynthesis framework with controllable GANs for the task of self-supervised object detection. We use collections of realworld images without bounding box annotations to learn to synthesize and detect objects. We leverage controllable GANs to synthesize images with pre-defined object properties and use them to train object detectors. We propose a tight end-to-end coupling of the synthesis and detection networks to optimally train our system. Finally, we also propose a method to optimally adapt SSOD to an intended target data without requiring labels for it. For the task of car detection, on the challenging KITTI and Cityscapes datasets, we show that SSOD outperforms the prior stateof-the-art purely image-based self-supervised object detection method Wetectron. Even without requiring any 3D CAD assets, it also surpasses the state-of-the-art renderingbased method Meta-Sim2. Our work advances the field of self-supervised object detection by introducing a successful new paradigm of using controllable GAN-based image synthesis for it and by significantly improving the baseline accuracy of the task. We open-source our code at https://github.com/NVlabs/SSOD.

show abstract

S‐LASSIE: Structure and smoothness enhanced learning from sparse image ensemble for 3D articulated shape reconstruction

Feng,

He,

Wang

et al. 2024

Computer Animation & Virtual

View full text Add to dashboard Cite

In computer vision, the task of 3D reconstruction from monocular sparse images poses significant challenges, particularly in the field of animal modelling. The diverse morphology of animals, their varied postures, and the variable conditions of image acquisition significantly complicate the task of accurately reconstructing their 3D shape and pose from a monocular image. To address these complexities, we propose S‐LASSIE, a novel technique for 3D reconstruction of quadrupeds from monocular sparse images. It requires only 10–30 images of similar breeds for training. To effectively mitigate depth ambiguities inherent in monocular reconstructions, S‐LASSIE employs a multi‐angle projection loss function. In addition, our approach, which involves fusion and smoothing of bone structures, resolves issues related to disjointed topological structures and uneven connections at junctions, resulting in 3D models with comprehensive topologies and improved visual fidelity. Our extensive experiments on the Pascal‐Part and LASSIE datasets demonstrate significant improvements in keypoint transfer, overall 2D IOU and visual quality, with an average keypoint transfer and overall 2D IOU of 59.6% and 86.3%, respectively, which are superior to existing techniques in the field.

show abstract

Canonical Surface Mapping via Geometric Cycle Consistency

Cited by 109 publications

References 43 publications

3D Reconstruction By Parameterized Surface Mapping

3D Reconstruction By Parameterized Surface Mapping

Self-Supervised Object Detection via Generative Image Synthesis

S‐LASSIE: Structure and smoothness enhanced learning from sparse image ensemble for 3D articulated shape reconstruction

Contact Info

Product

Resources

About