Point Cloud Deep Learning Network Based on Balanced Sampling and Hybrid Pooling

Abstract

The automatic semantic segmentation of point cloud data is important for applications in the fields of machine vision, virtual reality, and smart cities. The processing capability of the point cloud segmentation method with PointNet++ as the baseline needs to be improved for extremely imbalanced point cloud scenes. To address this problem, in this study, we designed a weighted sampling method based on farthest point sampling (FPS), which adjusts the sampling weight value according to the loss value of the model to equalize the sampling process. We also introduced the relational learning of the neighborhood space of the sampling center point in the feature encoding process, where the feature importance is distinguished by using a self-attention model. Finally, the global–local features were aggregated and transmitted using the hybrid pooling method. The experimental results of the six-fold crossover experiment showed that on the S3DIS semantic segmentation dataset, the proposed network achieved 9.5% and 11.6% improvement in overall point-wise accuracy (OA) and mean of class-wise intersection over union (MIoU), respectively, compared with the baseline. On the Vaihingen dataset, the proposed network achieved 4.2% and 3.9% improvement in OA and MIoU, respectively, compared with the baseline. Compared with the segmentation results of other network models on public datasets, our algorithm achieves a good balance between OA and MIoU.

PointNAC: Copula-Based Point Cloud Semantic Segmentation Network

“…They have relatively weak learning capabilities regarding the scale and structural information of the spatial context of the sampling center point, which affects their segmentation accuracy. To address this issue, Deng C. et al [23] proposed a segmentation network called BSH-Net with balanced sampling and hybrid pooling. This network initially assigns initial weights to each class of points and adjusts the sampling strategy based on each iteration, ensuring that the network can learn from all classes of samples.…”

“…The weighted balanced sampling module and the diversity pooling module in Figure 2 are described in detail in the literature [23]. This section mainly introduces the local stereoscopic feature-encoding module.…”

“…The training data is created by concatenating five partitions, while the test set consists of urban block and villa areas. At the same time, we also use the mean of class-wise intersection over union (MIoU), overall point-wise accuracy (OA), and balanced F score (F1 score) used by BSH-Net [23] for performance analysis and comparison. The calculation formulas as shown in Equation ( 8):…”

PointMM: Point Cloud Semantic Segmentation CNN under Multi-Spatial Feature Encoding and Multi-Head Attention Pooling

“…As a result, the overall segmentation accuracy is compromised, reaching only 82.1%. The learning ability of BSH-Net [34] for features of minority class samples is weak, resulting in an unsatisfactory average F1 score (69.5%). PointNAC builds upon the BSH-Net framework by introducing a 4D point pair feature encoding scheme, thereby enhancing the segmentation accuracy of the network.…”

“…Yin et al [33], based on geometric structure and object edge integrity, design a local feature encoding network using rapid point random sampling. In order to enhance a network's ability to learn local features, Deng et al [34] proposed PointNAC by introducing a point-pair feature encoding pattern and Copula correlation analysis module, and Wu et al [35] developed PointConv by introducing a novel weight calculation as well. Yan et al [36] designed an Adaptive Sampling Module and Local-Nonlocal (L-NL) Module based on attention mechanisms to mitigate noise and outliers that could disrupt the network's learning of local features.…”