A Prior Level Fusion Approach for the Semantic Segmentation of 3D Point Clouds Using Deep Learning

Abstract

Three-dimensional digital models play a pivotal role in city planning, monitoring, and sustainable management of smart and Digital Twin Cities (DTCs). In this context, semantic segmentation of airborne 3D point clouds is crucial for modeling, simulating, and understanding large-scale urban environments. Previous research studies have demonstrated that the performance of 3D semantic segmentation can be improved by fusing 3D point clouds and other data sources. In this paper, a new prior-level fusion approach is proposed for semantic segmentation of large-scale urban areas using optical images and point clouds. The proposed approach uses image classification obtained by the Maximum Likelihood Classifier as the prior knowledge for 3D semantic segmentation. Afterwards, the raster values from classified images are assigned to Lidar point clouds at the data preparation step. Finally, an advanced Deep Learning model (RandLaNet) is adopted to perform the 3D semantic segmentation. The results show that the proposed approach provides good results in terms of both evaluation metrics and visual examination with a higher Intersection over Union (96%) on the created dataset, compared with (92%) for the non-fusion approach.

DGPolarNet: Dynamic Graph Convolution Network for LiDAR Point Cloud Semantic Segmentation on Polar BEV

“…The points in the polar BEV grid defined as p i (r i , θ i , t i ) ∈ P are rasterized into a 3D array V (1) of size n 1 1 × n 1 2 × n 1 3 , which is then input into the FPS-KNN dynamic network as the first layer of the backbone network. For the first layer V (1) , n 1 1 is the batch size, and n 1 3 is the number of points in each batch. Each point has three attributes {r, θ, t}; thus, n 1 2 = 3.…”

“…Then, the vertices in 𝑉 ( ) and the edges in 𝐸 ( ) are input into the Conv2D and pooling operations to generate the output dataset 𝑉 ( 1) with a dimension of (𝑛 1…”

“…LiDAR sensors are essential devices for environmental perception tasks in smart vehicles, as they can scan millions of 3D points for each frame [1][2][3]. In recent years, LiDARbased semantic segmentation technology has achieved rapid development.…”

Weakly Supervised Semantic Segmentation of Point Cloud Scenes using Boundary-based Feature Aggregation

“…This is primarily due to the timeintensive and laborious properties of annotating point clouds. Existing methodologies for point cloud semantic segmentation in traditional contexts often encounter challenges when processing scene boundaries across diverse semantic categories [1][2][3]. A particularly demanding aspect involves the segmentation of scene boundaries, which requires distinguishing different semantic labels by sampling point features based on approximate color and geometric information.…”

Urban Aquatic Scene Expansion for Semantic Segmentation in Cityscapes

“…Machine learning plays a pivotal role in urban planning and development [1][2][3]. Particularly, semantic segmentation [4,5] can serve as a foundational technology in applications ranging from smart city design to environmental monitoring. This process involves classifying each pixel in an image to demarcate distinct regions with semantic relevance, thus facilitating a detailed understanding of urban landscapes [6,7].…”