Fitting Multiple Heterogeneous Models by Multi-Class Cascaded T-Linkage

Magri, Luca; Fusiello, Andrea

doi:10.1109/cvpr.2019.00764

Cited by 19 publications

(23 citation statements)

References 26 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-body motion. Provided point correspondences between two point clouds/images, rigid-body motion segmentation becomes a multi-model fitting problem, amenable for factorization techniques [21,42,78], graph optimization [47,36,12] or deep learning [40]. Among others, [79] handles raw scans and segments the rigidly moving parts using a Recurrent Neural Network (RNN).…”

Section: Related Workmentioning

confidence: 99%

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

Huang

Wang

Birdal

et al. 2021

Preprint

View full text Add to dashboard Cite

We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds. The two non-trivial challenges posed by this multi-scan multibody setting that we investigate are: (i) guaranteeing correspondence and segmentation consistency across multiple input point clouds capturing different spatial arrangements of bodies or body parts; and (ii) obtaining robust motionbased rigid body segmentation applicable to novel object categories.We propose an approach to address these issues that incorporates spectral synchronization into an iterative deep declarative network, so as to simultaneously recover consistent correspondences as well as motion segmentation. At the same time, by explicitly disentangling the correspondence and motion segmentation estimation modules, we achieve strong generalizability across different object categories. Our extensive evaluations demonstrate that our method is effective on various datasets ranging from rigid parts in articulated objects to individually moving objects in a 3D scene, be it single-view or full point clouds. Code at https://github.com/ huangjh-pub/multibody-sync.

show abstract

Section: Related Workmentioning

confidence: 99%

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

Huang

Wang

Birdal

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Several approaches [3,4,53,1] fit multiple instances of a model simultaneously by optimising an energy-based functional. Moreover, the algorithms in [3,4,43] approach the multi-class problem, i.e. when models of multiple types may fit the data.…”

Section: Related Workmentioning

confidence: 99%

Cuboids Revisited: Learning Robust 3D Shape Fitting to Single RGB Images

Kluger¹,

Ackermann²,

Brachmann³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans perceive and construct the surrounding world as an arrangement of simple parametric models. In particular, man-made environments commonly consist of volumetric primitives such as cuboids or cylinders. Inferring these primitives is an important step to attain high-level, abstract scene descriptions. Previous approaches directly estimate shape parameters from a 2D or 3D input, and are only able to reproduce simple objects, yet unable to accurately parse more complex 3D scenes. In contrast, we propose a robust estimator for primitive fitting, which can meaningfully abstract real-world environments using cuboids. A RANSAC estimator guided by a neural network fits these primitives to 3D features, such as a depth map. We condition the network on previously detected parts of the scene, thus parsing it one-by-one. To obtain 3D features from a single RGB image, we additionally optimise a feature extraction CNN in an end-to-end manner. However, naively minimising pointto-primitive distances leads to large or spurious cuboids occluding parts of the scene behind. We thus propose an occlusion-aware distance metric correctly handling opaque scenes. The proposed algorithm does not require labourintensive labels, such as cuboid annotations, for training. Results on the challenging NYU Depth v2 dataset demonstrate that the proposed algorithm successfully abstracts cluttered real-world 3D scene layouts.

show abstract

“…RANSACbased methods(e.g. [28] [9] [10] [35] [29] [11]) run revised RANSAC sequentially to obtain multiple model parameters. They change the sampling weight of each point in each iteration to get different model parameters.…”

Section: Related Workmentioning

confidence: 99%

“…However, this approach needs to train a detector or a segementation network [39] [25] for specific objects or classes, which does not apply to unknown objects or ar-bitrary 3D scans. Another solution is via multi-model fitting [32] [33] [34] or [35] [29] [11]. Existing multi-model fitting methods rely on sampling valid hypotheses, which involves a large number of sampling steps when the number of models or the outlier ratio becomes high, making the efficiency and robustness of those algorithms drop drastically.…”

Section: Introductionmentioning

confidence: 99%

Multi-instance Point Cloud Registration by Efficient Correspondence Clustering

Tang¹,

Zou²

2021

Preprint

View full text Add to dashboard Cite

We address the problem of estimating the poses of multiple instances of the source point cloud within a target point cloud. Existing solutions require sampling a lot of hypotheses to detect possible instances and reject the outliers, whose robustness and efficiency degrade notably when the number of instances and outliers increase. We propose to directly group the set of noisy correspondences into different clusters based on a distance invariance matrix. The instances and outliers are automatically identified through clustering. Our method is robust and fast. We evaluated our method on both synthetic and real-world datasets. The results show that our approach can correctly register up to 20 instances with an F1 score of 90.46% in the presence of 70% outliers, which performs significantly better and at least 10× faster than existing methods.

show abstract

Fitting Multiple Heterogeneous Models by Multi-Class Cascaded T-Linkage

Cited by 19 publications

References 26 publications

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

Cuboids Revisited: Learning Robust 3D Shape Fitting to Single RGB Images

Multi-instance Point Cloud Registration by Efficient Correspondence Clustering

Contact Info

Product

Resources

About