Articulation-Aware Canonical Surface Mapping

Kulkarni, Nilesh; Gupta, Abhinav; Fouhey, David F.; Tulsiani, Shubham

doi:10.1109/cvpr42600.2020.00053

Cited by 80 publications

(99 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, Canonical Surface Mapping (CSM) [5] predicts a UV mapping from a single image onto a canonical model, trained entirely using self-supervision, by introducing a geometric cycle-consistency term. For Kulkarni et al [10], the same mapping is applied but the canonical surface mesh can deform given an articulation parameter, which allows shape alignment to an input image. Our work is inspired by these, but differs in two key ways.…”

Section: Related Workmentioning

confidence: 99%

3D Reconstruction By Parameterized Surface Mapping

Langlois

Fisher

Wang

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

We introduce an approach for computing a 3D mesh from one or more views of an object by establishing dense correspondences between pixels in the views and 3D locations on a learnable parameterized surface. We propose a multi-view shape encoder that can be jointly trained with the AtlasNet surface parameterization. The shape is further refined using a novel geometric cycle-consistency loss between the learnable parameterized surface and input views. We demonstrate the efficacy of our approach on the ShapeNet-COCO dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

3D Reconstruction By Parameterized Surface Mapping

Langlois

Fisher

Wang

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…A growing number of studies have tackled the reconstruction of category-specific, natural articulated objects with a particular kinematic structure, such as the human body and animals. Representative works rely on the use of category-specific template models as the shape and pose prior (Loper et al, 2015;Zuffi et al, 2017;Bogo et al, 2016;Zuffi et al, 2019;Kulkarni et al, 2020). Another body Figure 2: Model overview of PPD.…”

Section: Related Workmentioning

confidence: 99%

“…For the "revolute" part, we set B i = T (q i )R(s i , u i ), where R(•) denotes a homogeneous rotation matrix given the rotation representation, and s i and u i represent the axis-angle rotation around the axis u i by angle s i . In human shape reconstruction methods using template shape, its pose is initialized to be close to the real distribution to avoid the local minima (Kanazawa et al, 2018;Kulkarni et al, 2020). Inspired by these approaches, we parametrize the joint direction as…”

Section: Part Shape Representationmentioning

confidence: 99%

Unsupervised Pose-Aware Part Decomposition for 3D Articulated Objects

Kawana¹,

Mukuta²,

Harada³

2021

Preprint

View full text Add to dashboard Cite

Articulated objects exist widely in the real world. However, previous 3D generative methods for unsupervised part decomposition are unsuitable for such objects, because they assume a spatially fixed part location, resulting in inconsistent part parsing. In this paper, we propose PPD (unsupervised Pose-aware Part Decomposition) to address a novel setting that explicitly targets man-made articulated objects with mechanical joints, considering the part poses. We show that categorycommon prior learning for both part shapes and poses facilitates the unsupervised learning of (1) part decomposition with non-primitive-based implicit representation, and (2) part pose as joint parameters under single-frame shape supervision. We evaluate our method on synthetic and real datasets, and we show that it outperforms previous works in consistent part parsing of the articulated objects based on comparable part pose estimation performance to the supervised baseline.

show abstract

“…In ad-dition to numerous semantic-specific details, recognition in novel viewpoints via direct appearance synthesis is suboptimal: one may be sure of the presence of a rug behind a couch, but unsure of its particular color. Similarly, there have been advances in learning to infer 3D properties of scenes from image cues [20,46,63], or with differentiable rendering [10,29,38,50] and other methods for bypassing the need for direct 3D supervision [27,33,34,68]. However, these approaches do not connect to complex scene semantics; they primarily focus on single objects or small, less diverse 3D annotated datasets.…”

Section: Introductionmentioning

confidence: 99%

Recognizing Scenes from Novel Viewpoints

Qian¹,

Kirillov²,

Ravi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Humans can perceive scenes in 3D from a handful of 2D views. For AI agents, the ability to recognize a scene from any viewpoint given only a few images enables them to efficiently interact with the scene and its objects. In this work, we attempt to endow machines with this ability. We propose a model which takes as input a few RGB images of a new scene and recognizes the scene from novel viewpoints by segmenting it into semantic categories. All this without access to the RGB images from those views. We pair 2D scene recognition with an implicit 3D representation and learn from multi-view 2D annotations of hundreds of scenes without any 3D supervision beyond camera poses. We experiment on challenging datasets and demonstrate our model's ability to jointly capture semantics and geometry of novel scenes with diverse layouts, object types and shapes. 1

show abstract

Articulation-Aware Canonical Surface Mapping

Cited by 80 publications

References 23 publications

3D Reconstruction By Parameterized Surface Mapping

3D Reconstruction By Parameterized Surface Mapping

Unsupervised Pose-Aware Part Decomposition for 3D Articulated Objects

Recognizing Scenes from Novel Viewpoints

Contact Info

Product

Resources

About