Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport

Saleh, Mahdi; Wu, Shuncheng; Cosmo, Luca; Navab, Nassir; Busam, Benjamin; Tombari, Federico

doi:10.1109/cvpr52688.2022.01146

Cited by 13 publications

(7 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the awareness of spatial positions is missing in the cross-frame aggregation. Lepard [20] extends the non-rigid shape matching [34,38,42] to point clouds [28] and proposes a re-positioning module to alleviate the pose variations. REGTR [48] directly regresses the corresponding coordinates and registers point clouds in an end-to-end fashion.…”

Section: Related Workmentioning

confidence: 99%

“…A branch of handcrafted descriptors [8,14,41] aligns the input to a canonical representation according to an estimated local reference frame (LRF), while the others [11,30,31] mine the rotation-invariant components and encode them as the representation of the local geometry. Inspired by that, some deep learning-based methods [1,4,9,13,32,34,49] are designed to be intrinsically rotation-invariant to make the neural models focus on the pose-agnostic pure geometry. As a pioneer, PPF-FoldNet [9] consumes PPF-based patches and learns the descriptors using a FoldingNet [47]-based architecture without supervision.…”

Section: Related Workmentioning

confidence: 99%

“…As a pioneer, PPF-FoldNet [9] consumes PPF-based patches and learns the descriptors using a FoldingNet [47]-based architecture without supervision. LRF-based works [1,13,32,34] achieve rotation invariance by aligning their input to the defined canonical representation. YOHO [44] adopts a group of rotations to learn a rotation-equivariant feature group and further obtain the invariance via group pooling.…”

Section: Related Workmentioning

confidence: 99%

“…With the emergence of deep neural models for 3D point analysis, e.g., multilayer perceptrons (MLPs)-based like PointNet [25,26], convolutionsbased like KPConv [6,40], and the attention-based like PointTransformer [33,53], recent approaches [1,7,9,10,13,19,20,27,32,[48][49][50][51] propose to learn descriptors from raw points as an alternative to handcrafted features that are less robust to occlusion and noise. The majority of deep point matchers [7,10,19,20,27,34,48,[50][51][52] is sensitive to rotations. Consequently, their invariance to rotations must be obtained extrinsically via augmented training to ensure that the same geometry under different poses can be depicted similarly.…”

Section: Introductionmentioning

confidence: 99%

“…However, it lacks a highly-representative geometry encoder since it relies on PointNet [25], which accounts for an ineffective local geometry description. Moreover, as depicting the cross-frame spatial relationships is non-trivial, previous works [19,27,34,50] merely leverage the contextual features in the cross-frame context aggregation, which neglects the positional information. Although RIGA proposes to learn a rotation-invariant position representation by leveraging an additional PointNet, this simple design is hard to model the complex cross-frame positional relationships and leads to less distinctive descriptors.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Rotation-Invariant Transformer for Point Cloud Matching

Yu¹,

Zheng²,

Hou³

et al. 2023

Preprint

View full text Add to dashboard Cite

The intrinsic rotation invariance lies at the core of matching point clouds with handcrafted descriptors. However, it is widely despised by recent deep matchers that obtain the rotation invariance extrinsically via data augmentation. As the finite number of augmented rotations can never span the continuous SO(3) space, these methods usually show instability when facing rotations that are rarely seen. To this end, we introduce RoITr, a Rotation-Invariant Transformer to cope with the pose variations in the point cloud matching task. We contribute both on the local and global levels. Starting from the local level, we introduce an attention mechanism embedded with Point Pair Feature (PPF)-based coordinates to describe the poseinvariant geometry, upon which a novel attention-based encoder-decoder architecture is constructed. We further propose a global transformer with rotation-invariant crossframe spatial awareness learned by the self-attention mechanism, which significantly improves the feature distinctiveness and makes the model robust with respect to the low overlap. Experiments are conducted on both the rigid and non-rigid public benchmarks, where RoITr outperforms all the state-of-the-art models by a considerable margin in the low-overlapping scenarios. Especially when the rotations are enlarged on the challenging 3DLoMatch benchmark, RoITr surpasses the existing methods by at least 13 and 5 percentage points in terms of Inlier Ratio and Registration Recall, respectively. Code is publicly available 1 .

show abstract