Learning Multiview 3D Point Cloud Registration

Gojčič, Žan; Zhou, Caifa; Wegner, Jan Dirk; Guibas, Leonidas J.; Birdal, Tolga

doi:10.1109/cvpr42600.2020.00183

Cited by 141 publications

(103 citation statements)

References 60 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…We compare our method against point‐to‐point ICP [2], point‐to‐plane ICP [18], deep global registration (DGR) [19] and 3D multiview registration (3DMR) [5]. The former two methods are popular ICP‐based methods and the latter two are state‐of‐the‐art deep learning‐based registration approaches.…”

Section: Methodsmentioning

confidence: 99%

Method for registration of 3D shapes without overlap for known 3D priors

Munteanu

2021

Electronics Letters

View full text Add to dashboard Cite

In 3D registration of point clouds, the goal is to find an optimal transformation that aligns the input shapes, provided that they have some overlap. Existing methods suffer from performance degradation when the overlapping ratio between the neighbouring point clouds is small. So far, there is no existing method that can be adopted for aligning shapes with no overlap. In this letter, to the best of knowledge, the first method for the registration of 3D shapes without overlap, assuming that the shapes correspond to partial views of a known semi-rigid 3D prior is presented. The method is validated and compared to existing methods on FAUST, which is a known dataset used for human body reconstruction. Experimental results show that this approach can effectively align shapes without overlap. Compared to existing state-of-theart methods, this approach avoids iterative optimization and is robust to outliers and inherent inaccuracies induced by an initial rough alignment of the shapes. Introduction: 3D registration is a classical and fundamental problem for countless applications. Since commodity depth cameras become less expensive and more accurate, depth images play an increasingly important role in numerous tasks [1]. In order to obtain comprehensive information from 3D scenery, point clouds captured from multiple views need to be aligned. The well-established method is iterative closest point (ICP) [2] based on which a myriad of flavours have been proposed. In ICP, given a source shape and a target shape, the following steps are performed: (1) for each point in the source shape, identify the closest corresponding point in the target shape; (2) predict the transformation by minimizing the mean square Euclidean distance between these correspondences; (3) transform the source shape using the predicted transformation from step 2; (4) iterate the above steps until the mean square distance reaches a pre-defined threshold. ICP and its variants are the dominating methods for the task of 3D registration. However, ICP-based methods assume that the source and target shapes have been roughly aligned with a sufficient overlap.. Deep learning has shown its excellent ability to solve various problems which are difficult or impossible to address using traditional approaches. Recent research strives to explore 3D registration via deep learning [3], [4], [5], [6]. However, these methods are designed for shapes that partially overlap. In this letter, we present a novel deep learning method for 3D shape registration. Compared to the existing methods, the main advantage of our method is that we successfully handled the non-overlapping shape registration problem. We assume that the shapes correspond to partial views of a known semi-rigid 3D prior. This problem is impossible to be addressed using ICP due to the lack of point correspondences. This is addressed in this letter of which the main contributions can be summarized as follows:

show abstract

Section: Methodsmentioning

confidence: 99%

Method for registration of 3D shapes without overlap for known 3D priors

Munteanu

2021

Electronics Letters

View full text Add to dashboard Cite

show abstract

“…In Fig. 5, Phong reflection method [20] and Blender (a software for generating and creating 3D shapes) are applied to project a series of robust shape images compared with OpenGL, but the rendering speed is slow. The finely textured images that are obtained provide a detailed description of the shape feature.…”

Section: Methodsmentioning

confidence: 99%

“…Some methods gathered point‐to‐point information using histograms [15–17]. Neural networks appear to be a particularly important role [1, 2, 18–20]. By imitating the encoder–decoder form of the hourglass network, Klokov and Lempitsky [21] used kd‐tree to build the encoder network for the 3D point cloud set and restored the input layer by learning to adjust the decoder parameter.…”

Section: Related Workmentioning

confidence: 99%

Balanced principal component for 3D shape recognition using convolutional neural networks

Luo

Zhang

et al. 2020

IET image process

View full text Add to dashboard Cite

Currently, PCA (principal component analysis) is widely used in many neural networks and has become a crucial part of the convolutional neural network (CNN) feature extraction. However, whether PCA is suitable for this process remains to be elucidated. The authors proposed a new method called balanced principal component (BPC) that generates a balanced local feature and combines with CNN as a layer to cope with the fusion problem. Specifically, BPC layer includes regionalisation module and average compression PCA (AC‐PCA) module. First, they used regionalisation module to generate some sub‐region that focuses on the local feature in each view. Secondly, the AC‐PCA module is a computational process that enlarges the feature matrix by PCA and eventually compacts the matrix to a one‐dimensional (1D) vector by AC. Next, all 1D vectors are compacted by AC to obtain a multi‐dimensional balance. Finally, they designed this layer with an end‐to‐end trainable structure to promote the feature extraction task of CNN. They addressed 3D shapes using a projection method that is pre‐trained on ImageNet and migration learning on ModelNet dataset. By comparing with the state‐of‐the‐art network, they achieved a significant gain in performance of retrieval and classification tasks.

show abstract

“…Deep features have also been employed in the recent registration methods in [114,115,72,12,38,123]. These methods employ a similar approach as the classical approaches described in 2.3, in that they minimize a loss function that resembles (2.5).…”

Section: Feature Based Registrationmentioning

confidence: 99%

Learning Representations for Segmentation and Registration

Lawin

2021

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

I could not have made it through my Ph.D. studies without the help of my colleagues at CVL. First and foremost I would like to thank my supervisor Per-Erik Forssén for his friendliness and all the valuable guidance. Throughout, he has always been encouraging and open to discuss ideas. I have also enjoyed our supervision meetings, and often certain information has come to light during our discussions. Next, I want to thank my co-supervisor Michael Felsberg who first took me in as a Ph.D. student, or should I say, tricked me into it. Either way, I am glad that he gave me the chance at perusing a Ph.D. degree. I am very thankful for his support and for learning from his knowledge and experience. To both my supervisors I want to express my gratitude for giving me the freedom and the encouragement to explore different topics and ideas. I also want to thank all my collaborators and co-authors for sharing their brilliant ideas and for our interesting discussions. Martin Danelljan for his notation, for inviting me into exciting research projects and for pushing me into taking time-consuming math courses. Fahad Shahbaz Khan for his encouraging attitude and spreading of positive energy during the darkest deadline hours. Andreas Robinson for sharing his programming skills and for patiently helping me with software installations and computer issues. Goutam Bhat for his awesome figures, ability to improve the results and our discussions on football. I am very proud of what we achieved in our joint works. Moreover, I want to thank all of CVL, including past and current members, for the friendly atmosphere at the lab and all the interesting discussions during coffee breaks. I further want to thank Zahra for arranging barbecues and Mikael for hosting board gaming sessions. Hannes, Abdo and Emanuel for their nice company while sharing offices with me. I am also grateful to everyone who joined me for late lunch, even if the only acceptable option was the bad house. Special thanks go to Per-Erik, Gustav, Abdo, and Andreas for helping out with the proofreading of the thesis manuscript. Lastly, I want to thank my family and all my friends for all the support and good times.

show abstract

Learning Multiview 3D Point Cloud Registration

Cited by 141 publications

References 60 publications

Method for registration of 3D shapes without overlap for known 3D priors

Method for registration of 3D shapes without overlap for known 3D priors

Balanced principal component for 3D shape recognition using convolutional neural networks

Learning Representations for Segmentation and Registration

Contact Info

Product

Resources

About