Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network

Zhao, Ran; Pang, Muye; Wang, Jidong

doi:10.1080/13658816.2018.1431840

Cited by 118 publications

(105 citation statements)

References 32 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…Convolutional neural networks (CNNs) have achieved great success on various 2D image recognition tasks, including scene classification, object detection, semantic segmentation, and many others. Following the success of 2D CNNs, many studies have tried to generate feature images from point clouds and then employ convolutional neural networks to achieve airborne LiDAR point cloud classification (Yang et al, 2017b(Yang et al, ,b, 2018Zhao et al, 2018 (Zhao et al, 2018).…”

Section: Feature Image Based Classification Methodsmentioning

confidence: 99%

Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification

Wen

Yang

et al. 2020

ISPRS Journal of Photogrammetry and Remote Sensing

View full text Add to dashboard Cite

Point cloud classification plays an important role in a wide range of airborne light detection and ranging (LiDAR) applications, such as topographic mapping, forest monitoring, power line detection, and road detection. However, due to the sensor noise, high redundancy, incompleteness, and complexity of airborne LiDAR systems, point cloud classification is challenging. Traditional point cloud classification methods mostly focus on the development of handcrafted point geometry features and employ machine learning-based classification models to conduct point classification. In recent years, the advances of deep learning models have caused researchers to shift their focus towards machine learning-based models, specifically deep neural networks, to classify airborne LiDAR point clouds. These learning-based methods start by transforming the unstructured 3D point sets to regular 2D representations, such as collections of feature images, and then employ a 2D CNN for point classification. Moreover, these methods usually need to calculate additional local geometry features, such as planarity, sphericity and roughness, to make use of the local structural information in the original 3D space. Nonetheless, the 3D to 2D conversion results in information loss. In this paper, we proposed a directionally constrained fully convolutional neural network (D-FCN) that can take the original 3D coordinates and LiDAR intensity as input; thus, it can directly apply to unstructured 3D point clouds for semantic labeling. Specifically, we first introduce a novel directionally constrained point convolution (D-Conv) module to extract locally representative features of 3D point sets from the projected 2D receptive fields. To make full use of the orientation information of neighborhood points, the proposed D-Conv module performs convolution in an orientation-aware manner by using a directionally constrained nearest neighborhood search. Then, we designed a multiscale fully convolutional neural network with downsampling and upsampling blocks to enable multiscale point feature learning. The proposed D-FCN model can therefore process input point cloud with arbitrary sizes and directly predict the semantic labels for all the input points in an end-to-end manner. Without involving additional geometry features as input, the proposed method has demonstrated superior performance on the International Society for Photogrammetry and Remote Sensing (ISPRS) 3D labeling benchmark dataset. The results show that our model has achieved a new state-of-the-art level of performance with an average F1 score of 70.7%, and it has improved the performance by a large margin on categories with a small number of points (such as powerline, car, and facade).

show abstract

Section: Feature Image Based Classification Methodsmentioning

confidence: 99%

Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification

Wen

Yang

et al. 2020

ISPRS Journal of Photogrammetry and Remote Sensing

View full text Add to dashboard Cite

show abstract

“…We compare our method with the result provided in Reference [26] and the submitted results with published papers provided by the ISPRS Semantic Labeling Benchmark. Reference [47,48,5] adopted the traditional machine learning classifiers to classify ALS point clouds, while Reference [49][50][51][52] leveraged deep learning for the semantic classification. For the sake of clarity and readability, the results achieved by each research group and our model (namely LCS-CRF) are listed for comparison in Table 3.…”

Section: Quantitative Comparisonmentioning

confidence: 99%

Large-Scale ALS Data Semantic Classification Integrating Location-Context-Semantics Cues by Higher-Order CRF

Han

Wang

Huang

et al. 2020

Sensors

View full text Add to dashboard Cite

We designed a location-context-semantics-based conditional random field (LCS-CRF) framework for the semantic classification of airborne laser scanning (ALS) point clouds. For ALS datasets of high spatial resolution but with severe noise pollutions, more contexture and semantics cues, besides location information, can be exploited to surmount the decrease of discrimination of features for classification. This paper mainly focuses on the semantic classification of ALS data using mixed location-context-semantics cues, which are integrated into a higher-order CRF framework by modeling the probabilistic potentials. The location cues modeled by the unary potentials can provide basic information for discriminating the various classes. The pairwise potentials consider the spatial contextual information by establishing the neighboring interactions between points to favor spatial smoothing. The semantics cues are explicitly encoded in the higher-order potentials. The higher-order potential operates at the clusters level with similar geometric and radiometric properties, guaranteeing the classification accuracy based on semantic rules. To demonstrate the performance of our approach, two standard benchmark datasets were utilized. Experiments show that our method achieves superior classification results with an overall accuracy of 83.1% on the Vaihingen Dataset and an overall accuracy of 94.3% on the Graphics and Media Lab (GML) Dataset A compared with other classification algorithms in the literature. 2 of 28 from different equipment. Therefore, we incorporate location, spatial contextual, and semantics cues within a higher-order conditional random field (CRF) framework to provide complementary information from varying perspectives, so that it can address the common misjudgment of semantic classes in ALS point clouds, from the perspectives of the accuracy of each class and the overall accuracy. Related Works for ALS Point Cloud ClassificationAccording to the type of entity used for classification, existing methods can be categorized as point-based and cluster-based (or segment-based) [2,3]. Point-based methods classify each point of the ALS data by using features as the inputs for supervised or unsupervised classifiers [4], while cluster-based methods segment the ALS data into clusters, then class labels are assigned to the clusters in which all points share the same class label [5,6]. We briefly review the aforementioned methods, and demonstrate the rationale for our method in what follows.Point-based methods generally extract point-wise features locally from the neighborhood defined by a sphere or cylinder. Therefore, such methods usually focus on the selection of discriminative features and effective classifiers. For instance, Reference [7] worked on 3D scene analysis, including geometric features extraction and optimal neighbors selection. Then an optimal eigenentropy-based scale selection method was proposed. Reference [2] combined airborne LiDAR with images to extract more discriminative features. Then, based on these ...

show abstract

“…pedestrians, cyclists) in 3D point clouds based on the encoding of point clouds into equally spaced 3D voxels. Zhao et al (2018) classified ALS point clouds via deep features learned by a multi-scale CNN. The method creates a group of multi-scale contextual images for each 3D point and is ranked first on the ISPRS benchmark dataset (ISPRS, 2019).…”

Section: D and 3d Dnnsmentioning

confidence: 99%

Semantic Labeling of Als Point Clouds for Tree Species Mapping Using the Deep Neural Network Pointnet++

Briechle

Krzystek

Vosselman

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90% in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision = 93%, recall = 80%) and deciduous trees (precision = 82%, recall = 92%). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batchesà 400 m 2 . The achieved per-point classification results after 163 training epochs for coniferous trees (precision = 90%, recall = 79%) and deciduous trees (precision = 81%, recall = 91%) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.

show abstract

Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network

Cited by 118 publications

References 32 publications

Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification

Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification

Large-Scale ALS Data Semantic Classification Integrating Location-Context-Semantics Cues by Higher-Order CRF

Semantic Labeling of Als Point Clouds for Tree Species Mapping Using the Deep Neural Network Pointnet++

Contact Info

Product

Resources

About