Superpoint Network for Point Cloud Oversegmentation

Hui, Le; Jia, Yuan; Cheng, Mingmei; Xie, Jin; Zhang, Xiaoya; Yang, Jian

doi:10.1109/iccv48922.2021.00546

Cited by 28 publications

(17 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing deep learning-based methods for processing 3D data are limited by the data size, especially for large-scale urban scenes. This has already motivated over-segmentation for semantic segmentation [18,22,23,52]. Following the spirit of the previous work for improving efficiency, our over-segmentation facilitates better object boundaries and strengthens semantic segmentation by distinctive local and non-local features, which is suitable for the subsequent classification using graph convolutional networks (GCN).…”

Section: Introductionmentioning

confidence: 84%

“…Many methods for over-segmentation of 3D data are inspired by image over-segmentation algorithms [28] and can be divided into four categories, namely, primitivebased fitting [21,41,49], graph-based partitioning [1,23], local region expansion [6,21,27,29,34,34,39,50], and learning-based methods [18,22]. Over-segmentation often serves as pre-processing for tasks such as semantic segmentation, instance segmentation, or reconstruction, to reduce the complexity of subsequent tasks by using fewer segments with local homogeneity.…”

Section: Over-segmentation Of 3d Datamentioning

confidence: 99%

“…Due to the complexity of real-world scenes and the irregularity of the data, it is challenging to obtain over-segmentation results with a desired number of segments and clear object boundaries. The aforementioned methods are either limited by the primitive types (e.g., plane, sphere, and cylinder), or suffering from severe under-segmentation errors when the number of segments is reduced, or by the type of available labels in the training data (e.g., a few methods require instance labels [18,22]). We propose to partition the input meshes into a relatively small number of homogeneous regions with clear object boundaries based on both geometric and photometric characteristics, which is beneficial to semantic segmentation.…”

Section: Over-segmentation Of 3d Datamentioning

confidence: 99%

See 2 more Smart Citations

PSSNet: Planarity-sensible Semantic Segmentation of Large-scale Urban Meshes

Gao¹,

Nan²,

Boom³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a novel deep learning-based framework to interpret 3D urban scenes represented as textured meshes. Based on the observation that object boundaries typically align with the boundaries of planar regions, our framework achieves semantic segmentation in two steps: planaritysensible over-segmentation followed by semantic classification. The over-segmentation step generates an initial set of mesh segments that capture the planar and non-planar regions of urban scenes. In the subsequent classification step, we construct a graph that encodes geometric and photometric features of the segments in its nodes and multiscale contextual features in its edges. The final semantic segmentation is obtained by classifying the segments using a graph convolutional network. Experiments and comparisons on a large semantic urban mesh benchmark demonstrate that our approach outperforms the state-ofthe-art methods in terms of boundary quality and mean IoU (intersection over union). Besides, we also introduce several new metrics for evaluating mesh over-segmentation methods dedicated for semantic segmentation, and our proposed over-segmentation approach outperforms stateof-the-art methods on all metrics. Our source code will be released when the paper is accepted.

show abstract

Section: Introductionmentioning

confidence: 84%

Section: Over-segmentation Of 3d Datamentioning

confidence: 99%

Section: Over-segmentation Of 3d Datamentioning

confidence: 99%

See 1 more Smart Citation

PSSNet: Planarity-sensible Semantic Segmentation of Large-scale Urban Meshes

Gao¹,

Nan²,

Boom³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It will be meaningful to present an adaptive sampling method based on the self-attention mechanism for the hierarchical Transformer. Additionally, inspired by the superpixel [138] in the 2D field, we argue that it is feasible to utilize the attention map in 3D Transformers to obtain the "superpoint" [139] for point cloud oversegmentation, converting point-level 3D data into districtlevel data. In this way, this adaptive clustering technique can be used to replace the query ball grouping method.…”

Section: A Discussionmentioning

confidence: 99%

Transformers in 3D Point Clouds: A Survey

Lu¹,

Quan²,

Wei³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, Transformer models have been proven to have the remarkable ability of long-range dependencies modeling. They have achieved satisfactory results both in Natural Language Processing (NLP) and image processing. This significant achievement sparks great interest among researchers in 3D point cloud processing to apply them to various 3D tasks. Due to the inherent permutation invariance and strong global feature learning ability, 3D Transformers are well suited for point cloud processing and analysis. They have achieved competitive or even better performance compared to the state-of-the-art non-Transformer algorithms. This survey aims to provide a comprehensive overview of 3D Transformers designed for various tasks (e.g. point cloud classification, segmentation, object detection, and so on). We start by introducing the fundamental components of the general Transformer and providing a brief description of its application in 2D and 3D fields. Then, we present three different taxonomies (i.e., Transformer implementation-based taxonomy, data representation-based taxonomy, and task-based taxonomy) for method classification, which allows us to analyze involved methods from multiple perspectives. Furthermore, we also conduct an investigation of 3D self-attention mechanism variants designed for performance improvement. To demonstrate the superiority of 3D Transformers, we compare the performance of Transformer-based algorithms in terms of point cloud classification, segmentation, and object detection. Finally, we point out three potential future research directions, expecting to provide some benefit references for the development of 3D Transformers.

show abstract

“…At the same time, according to global energy optimization, the superpoint [20] was constructed by geometrically and even physically partitioning the point clouds without predefining the number of segments, which minimizes unnecessary segmentation of objects with large areas but maintains topological relationships among superpoints by building a global graph. Later, the construction of superpoints was improved by building a label consistency loss between true labels of points and pseudo labels of the superpoints in an end-to-end network [34]. Furthermore, the cascaded nonlocal network [35] adopted the superpoints as basic units and built a nonlocal operation with three granularity levels, including neighborhood-level, superpoint-level, and global-level.…”

Section: Deep Learning Network For Semantic Segmentationmentioning

confidence: 99%

IAGC: Interactive Attention Graph Convolution Network for Semantic Segmentation of Point Clouds in Building Indoor Environment

Zhai

Zou

et al. 2022

IJGI

View full text Add to dashboard Cite

Point-based networks have been widely used in the semantic segmentation of point clouds owing to the powerful 3D convolution neural network (CNN) baseline. Most of the current methods resort to intermediate regular representations for reorganizing the structure of point clouds for 3D CNN networks, but they may neglect the inherent contextual information. In our work, we focus on capturing discriminative features with the interactive attention mechanism and propose a novel method consisting of the regional simplified dual attention network and global graph convolution network. Firstly, we cluster homogeneous points into superpoints and construct a superpoint graph to effectively reduce the computation complexity and greatly maintain spatial topological relations among superpoints. Secondly, we integrate cross-position attention and cross-channel attention into a single head attention module and design a novel interactive attention gating (IAG)-based multilayer perceptron (MLP) network (IAG–MLP), which is utilized for the expansion of the receptive field and augmentation of discriminative features in local embeddings. Afterwards, the combination of stacked IAG–MLP blocks and the global graph convolution network, called IAGC, is proposed to learn high-dimensional local features in superpoints and progressively update these local embeddings with the recurrent neural network (RNN) network. Our proposed framework is evaluated on three indoor open benchmarks, and the 6-fold cross-validation results of the S3DIS dataset show that the local IAG–MLP network brings about 1% and 6.1% improvement in overall accuracy (OA) and mean class intersection-over-union (mIoU), respectively, compared with the PointNet local network. Furthermore, our IAGC network outperforms other CNN-based approaches in the ScanNet V2 dataset by at least 7.9% in mIoU. The experimental results indicate that the proposed method can better capture contextual information and achieve competitive overall performance in the semantic segmentation task.

show abstract

Superpoint Network for Point Cloud Oversegmentation

Cited by 28 publications

References 35 publications

PSSNet: Planarity-sensible Semantic Segmentation of Large-scale Urban Meshes

PSSNet: Planarity-sensible Semantic Segmentation of Large-scale Urban Meshes

Transformers in 3D Point Clouds: A Survey

IAGC: Interactive Attention Graph Convolution Network for Semantic Segmentation of Point Clouds in Building Indoor Environment

Contact Info

Product

Resources

About