3D Point Cloud Recognition Based on a Multi-View Convolutional Neural Network

Zhang, Le; Sun, Jian; Zheng, Qiang

doi:10.3390/s18113681

Cited by 36 publications

(22 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-view-based methods [45,[52][53][54][55][56][57][58] take advantage of the benefits of the already matured 2D CNNs and apply them into three dimensions. Because images are actual representations of the 3D world squashed onto a 2D grid by a camera, methods in this category follow this technique by converting point cloud data into a collection of 2D images and applying existing 2D CNN techniques to it; see Figure 5.…”

Section: Multi-view-based Approachmentioning

confidence: 99%

Review: Deep Learning on 3D Point Clouds

et al. 2020

View full text Add to dashboard Cite

A point cloud is a set of points defined in a 3D metric space. Point clouds have become one of the most significant data formats for 3D representation and are gaining increased popularity as a result of the increased availability of acquisition devices, as well as seeing increased application in areas such as robotics, autonomous driving, and augmented and virtual reality. Deep learning is now the most powerful tool for data processing in computer vision and is becoming the most preferred technique for tasks such as classification, segmentation, and detection. While deep learning techniques are mainly applied to data with a structured grid, the point cloud, on the other hand, is unstructured. The unstructuredness of point clouds makes the use of deep learning for its direct processing very challenging. This paper contains a review of the recent state-of-the-art deep learning techniques, mainly focusing on raw point cloud data. The initial work on deep learning directly with raw point cloud data did not model local regions; therefore, subsequent approaches model local regions through sampling and grouping. More recently, several approaches have been proposed that not only model the local regions but also explore the correlation between points in the local regions. From the survey, we conclude that approaches that model local regions and take into account the correlation between points in the local regions perform better. Contrary to existing reviews, this paper provides a general structure for learning with raw point clouds, and various methods were compared based on the general structure. This work also introduces the popular 3D point cloud benchmark datasets and discusses the application of deep learning in popular 3D vision tasks, including classification, segmentation, and detection.

show abstract

Section: Multi-view-based Approachmentioning

confidence: 99%

Review: Deep Learning on 3D Point Clouds

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Many state-of-the-art methods have been developed that process point cloud directly despite the challenging aspect of its unstructured and irregular format [4,32,[34][35][36][37][38]. Other approaches convert the point cloud data into a structured form [50][51][52][53], and our research goes in the same direction. We have developed an approach based on an indirect process.…”

Section: Discussionmentioning

confidence: 96%

3D Point Cloud Semantic Augmentation: Instance Segmentation of 360° Panoramas by Deep Learning Techniques

Karara¹,

Hajji²,

Poux

2021

Remote Sensing

View full text Add to dashboard Cite

Semantic augmentation of 3D point clouds is a challenging problem with numerous real-world applications. While deep learning has revolutionised image segmentation and classification, its impact on point cloud is an active research field. In this paper, we propose an instance segmentation and augmentation of 3D point clouds using deep learning architectures. We show the potential of an indirect approach using 2D images and a Mask R-CNN (Region-Based Convolution Neural Network). Our method consists of four core steps. We first project the point cloud onto panoramic 2D images using three types of projections: spherical, cylindrical, and cubic. Next, we homogenise the resulting images to correct the artefacts and the empty pixels to be comparable to images available in common training libraries. These images are then used as input to the Mask R-CNN neural network, designed for 2D instance segmentation. Finally, the obtained predictions are reprojected to the point cloud to obtain the segmentation results. We link the results to a context-aware neural network to augment the semantics. Several tests were performed on different datasets to test the adequacy of the method and its potential for generalisation. The developed algorithm uses only the attributes X, Y, Z, and a projection centre (virtual camera) position as inputs.

show abstract

“…Deep learning-based classification is divided into five categories based on different point cloud representations: Multi-view image, 2.5D DSM, voxel, raw point cloud and point cloud graph. Among the five categories, multi-view image, 2.5D DSM and voxel-based methods are indirect methods, where the Convolutional Neural Networks (CNNs) are working on multi-view images [25,26], voxels [27] or 2.5D [28,29] data rather than raw point clouds. The classification results are then transformed to the raw point clouds.…”

Section: Point Cloud Classificationmentioning

confidence: 99%

An Object-Based Bidirectional Method for Integrated Building Extraction and Change Detection between Multimodal Point Clouds

2020

View full text Add to dashboard Cite

Building extraction and change detection are two important tasks in the remote sensing domain. Change detection between airborne laser scanning data and photogrammetric data is vulnerable to dense matching errors, mis-alignment errors and data gaps. This paper proposes an unsupervised object-based method for integrated building extraction and change detection. Firstly, terrain, roofs and vegetation are extracted from the precise laser point cloud, based on “bottom-up” segmentation and clustering. Secondly, change detection is performed in an object-based bidirectional manner: Heightened buildings and demolished buildings are detected by taking the laser scanning data as reference, while newly-built buildings are detected by taking the dense matching data as reference. Experiments on two urban data sets demonstrate its effectiveness and robustness. The object-based change detection achieves a recall rate of 92.31% and a precision rate of 88.89% for the Rotterdam dataset; it achieves a recall rate of 85.71% and a precision rate of 100% for the Enschede dataset. It can not only extract unchanged building footprints, but also assign heightened or demolished labels to the changed buildings.

show abstract

3D Point Cloud Recognition Based on a Multi-View Convolutional Neural Network

Cited by 36 publications

References 21 publications

Review: Deep Learning on 3D Point Clouds

Review: Deep Learning on 3D Point Clouds

3D Point Cloud Semantic Augmentation: Instance Segmentation of 360° Panoramas by Deep Learning Techniques

An Object-Based Bidirectional Method for Integrated Building Extraction and Change Detection between Multimodal Point Clouds

Contact Info

Product

Resources

About