A Review on Deep Learning Techniques for 3D Sensed Data Classification

Griffiths, David; Boehm, J.

doi:10.3390/rs11121499

Cited by 161 publications

(98 citation statements)

References 115 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Much like with 2D image understanding, 3D understanding has greatly benefited from the current technological surge in the field of deep learning. With applications ranging from remote sensing, mapping, monitoring, city modeling, it is clear why robust and autonomous information extraction from 3D data is in high demand [19]. In this research unlike the traditional methods which transform 3D point clouds to DSM data for modeling, we propose a 3D point cloud segmentation method by fusing features from 2D aerial image.…”

Section: Discussionmentioning

confidence: 99%

“…This data transformation renders unnecessarily volume in the resulting data and also introducing quantization artifacts that can obscure such as in case of interpolation [16]. Point cloud processing in a deep learning network without generating regular data format is considered a challenging task [19]. In light of the irregular format of point cloud data, a novel approach, named PointNet that consumes raw point cloud directly has proposed by Qi et al [20].…”

Section: D Point Cloud Segmentation Approachesmentioning

confidence: 99%

See 1 more Smart Citation

A Point-Wise LiDAR and Image Multimodal Fusion Network (PMNet) for Aerial Point Cloud 3D Semantic Segmentation

2019

View full text Add to dashboard Cite

3D semantic segmentation of point cloud aims at assigning semantic labels to each point by utilizing and respecting the 3D representation of the data. Detailed 3D semantic segmentation of urban areas can assist policymakers, insurance companies, governmental agencies for applications such as urban growth assessment, disaster management, and traffic supervision. The recent proliferation of remote sensing techniques has led to producing high resolution multimodal geospatial data. Nonetheless, currently, only limited technologies are available to fuse the multimodal dataset effectively. Therefore, this paper proposes a novel deep learning-based end-to-end Point-wise LiDAR and Image Multimodal Fusion Network (PMNet) for 3D segmentation of aerial point cloud by fusing aerial image features. PMNet respects basic characteristics of point cloud such as unordered, irregular format and permutation invariance. Notably, multi-view 3D scanned data can also be trained using PMNet since it considers aerial point cloud as a fully 3D representation. The proposed method was applied on two datasets (1) collected from the urban area of Osaka, Japan and (2) from the University of Houston campus, USA and its neighborhood. The quantitative and qualitative evaluation shows that PMNet outperforms other models which use non-fusion and multimodal fusion (observational-level fusion and feature-level fusion) strategies. In addition, the paper demonstrates the improved performance of the proposed model (PMNet) by over-sampling/augmenting the medium and minor classes in order to address the class-imbalance issues.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: D Point Cloud Segmentation Approachesmentioning

confidence: 99%

A Point-Wise LiDAR and Image Multimodal Fusion Network (PMNet) for Aerial Point Cloud 3D Semantic Segmentation

2019

View full text Add to dashboard Cite

show abstract

“…For a further reading on point cloud segmentation and classification readers may refer to review articles of Nguyen and Le (2013) and Grilli et al (2017). For a detailed review on deep learning studies on 3D data (including point clouds and RGB-Depth data) classification, readers may refer to review article by Griffiths and Boehm (2019).…”

Section: Classification Approaches Based On Artificial Intelligence Mmentioning

confidence: 99%

Aerial Point Cloud Classification With Deep Learning and Machine Learning Algorithms

Özdemir

Remondino

Golkar

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

With recent advances in technology, 3D point clouds are getting more and more frequently requested and used, not only for visualization needs but also e.g. by public administrations for urban planning and management. 3D point clouds are also a very frequent source for generating 3D city models which became recently more available for many applications, such as urban development plans, energy evaluation, navigation, visibility analysis and numerous other GIS studies. While the main data sources remained the same (namely aerial photogrammetry and LiDAR), the way these city models are generated have been evolving towards automation with different approaches. As most of these approaches are based on point clouds with proper semantic classes, our aim is to classify aerial point clouds into meaningful semantic classes, e.g. ground level objects (GLO, including roads and pavements), vegetation, buildings' facades and buildings' roofs. In this study we tested and evaluated various machine learning algorithms for classification, including three deep learning algorithms and one machine learning algorithm. In the experiments, several hand-crafted geometric features depending on the dataset are used and, unconventionally, these geometric features are used also for deep learning. Figure 1. A LiDAR point cloud classified with our approach in 5 classes: buildings (red), powerline poles (orange), powerline cables (black), ground/soil (light green) and trees (dark green).

show abstract

“…These efforts, that encouraged developers and users to deliver comparative performance analyses on the provided datasets, are part of a broader landscape of benchmark actions, that have been assessing the performance of image-based 3D geometry reconstruction and scene classification algorithms over the last twenty years. An overview of these state-of-the-art initiatives undertaken by both the computer vision community and the geospatial one is given in Table 1, while a complete review of publicly available benchmark datasets for deep learning evaluation is discussed by Griffiths and Boehm (2019a).…”

Section: Related Benchmark Activitiesmentioning

confidence: 99%

A Multi-Purpose Benchmark for Photogrammetric Urban 3d Reconstruction in a Controlled Environment

Özdemir

Toschi

Remondino

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. This paper presents a novel multi-purpose benchmark for assessing the performance of the entire image-based pipeline for 3D urban reconstruction and 3D data classification. The proposed benchmark is called “3DOMcity” photogrammetric contest and is publicly available at https://3dom.fbk.eu/3domcity-benchmark. Compared to previous benchmark initiatives, the innovative aspects introduced by 3DOMcity are threefold. First, it covers the entire 3D reconstruction pipeline, including image orientation, dense image matching, point cloud classification and 3D building reconstruction. Second, the performance assessment is performed within a metrological context. Finally, the provided datasets offer 2D and 3D data collected at a very high spatial resolution. Furthermore, the benchmark is multi-purpose since it involves a multiplicity of tasks, that can be either performed independently from each other’s, or grouped together. The contributions of this paper are (i) the presentation of the novel benchmark dataset and corresponding reference data, (ii) the description of the multi-purpose benchmark tasks and evaluation procedures, and (iii) the discussion of first results of image orientation, dense image matching and classification.

show abstract

A Review on Deep Learning Techniques for 3D Sensed Data Classification

Cited by 161 publications

References 115 publications

A Point-Wise LiDAR and Image Multimodal Fusion Network (PMNet) for Aerial Point Cloud 3D Semantic Segmentation

A Point-Wise LiDAR and Image Multimodal Fusion Network (PMNet) for Aerial Point Cloud 3D Semantic Segmentation

Aerial Point Cloud Classification With Deep Learning and Machine Learning Algorithms

A Multi-Purpose Benchmark for Photogrammetric Urban 3d Reconstruction in a Controlled Environment

Contact Info

Product

Resources

About