ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes

Sajnani, Rahul; Poulenard, Adrien; Jain, Jivitesh; Dua, Radhika; Guibas, Leonidas J.; Sridhar, Srinath

doi:10.1109/cvpr52688.2022.01646

Cited by 22 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, future work should consider moving beyond supervised learning of fields and consider weakly‐ or self‐supervised learning as an alternative. Building upon advances in 3D deep learning such as transformation equivariance [SPJ*22, SSM*20,STD*21] could allow neural fields to be data efficient and generalize better.…”

Section: Discussionmentioning

confidence: 99%

Neural Fields in Visual Computing and Beyond

Xie

Takikawa

Saito

et al. 2022

Computer Graphics Forum

Self Cite

231

View full text Add to dashboard Cite

Manipulation 2D and 3D Reconstruction Generative Models Digital Humans Compression 903.63 KB Robotics …and Beyond! Figure 1: Contribution of this report. Following a survey of over 250 papers, we provide a review of (Part I) techniques in neural fields such as prior learning and conditioning, representations, forward maps, architectures, and manipulation, and of (Part II) applications in visual computing including 2D image processing, 3D scene reconstruction, generative modeling, digital humans, compression, robotics, and beyond. This report is complemented by a community-driven website with search, filtering, bibliographic, and visualization features.

show abstract

Section: Discussionmentioning

confidence: 99%

Neural Fields in Visual Computing and Beyond

Xie

Takikawa

Saito

et al. 2022

Computer Graphics Forum

Self Cite

231

View full text Add to dashboard Cite

show abstract

“…For example, equivariant networks with SE(3)-equivariance are developed to process 3D data such as point clouds [10], meshes [13], and voxels [35]. However, due to the added complexity, most equivariant networks for 3D perception are restricted to relatively simple tasks with small-scale inputs, such as object-wise classification, registration, part segmentation, and reconstruction [30,46,9]. In the following, we will review recent progress in extending equivariance to large-scale outdoor 3D perception tasks.…”

Section: Equivariant Learningmentioning

confidence: 99%

4D Panoptic Segmentation as Invariant and Equivariant Field Prediction

Zhu¹,

Han²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this paper, we develop rotation-equivariant neural networks for 4D panoptic segmentation. 4D panoptic segmentation is a recently established benchmark task for autonomous driving, which requires recognizing semantic classes and object instances on the road based on LiDAR scans, as well as assigning temporally consistent IDs to instances across time. We observe that the driving scenario is symmetric to rotations on the ground plane. Therefore, rotation-equivariance could provide better generalization and more robust feature learning. Specifically, we review the object instance clustering strategies, and restate the centerness-based approach and the offset-based approach as the prediction of invariant scalar fields and equivariant vector fields. Other sub-tasks are also unified from this perspective, and different invariant and equivariant layers are designed to facilitate their predictions. Through evaluation on the standard 4D panoptic segmentation benchmark of SemanticKITTI, we show that our equivariant models achieve higher accuracy with lower computational costs compared to their non-equivariant counterparts. Moreover, our method sets the new state-of-the-art performance and achieves 1st place on the SemanticKITTI 4D Panoptic Segmentation leaderboard.

show abstract

“…Siamese training enables feature disentanglement, which is an efficient technique in exploring the data variation and similarity. Recent works employ siamese training in disentangling rotation-equivariant and rotation-invariant features (Sun et al 2021;Gu et al 2020;Chen, Yang, and Tao 2022;Sajnani et al 2022). However, those methods mainly focus on registration and reconstruction tasks, and the extracted equivariant features are used for canonicalization.…”

Section: Related Workmentioning

confidence: 99%

PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration

Zhang

et al. 2023

AAAI

View full text Add to dashboard Cite

Recent interest in point cloud analysis has led rapid progress in designing deep learning methods for 3D models. However, state-of-the-art models are not robust to rotations, which remains an unknown prior to real applications and harms the model performance. In this work, we introduce a novel Patch-wise Rotation-invariant network (PaRot), which achieves rotation invariance via feature disentanglement and produces consistent predictions for samples with arbitrary rotations. Specifically, we design a siamese training module which disentangles rotation invariance and equivariance from patches defined over different scales, e.g., the local geometry and global shape, via a pair of rotations. However, our disentangled invariant feature loses the intrinsic pose information of each patch. To solve this problem, we propose a rotation-invariant geometric relation to restore the relative pose with equivariant information for patches defined over different scales. Utilising the pose information, we propose a hierarchical module which implements intra-scale and inter-scale feature aggregation for 3D shape learning. Moreover, we introduce a pose-aware feature propagation process with the rotation-invariant relative pose information embedded. Experiments show that our disentanglement module extracts high-quality rotation-robust features and the proposed lightweight model achieves competitive results in rotated 3D object classification and part segmentation tasks.

show abstract

ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes

Cited by 22 publications

References 31 publications

Neural Fields in Visual Computing and Beyond

Neural Fields in Visual Computing and Beyond

4D Panoptic Segmentation as Invariant and Equivariant Field Prediction

PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration

Contact Info

Product

Resources

About