PVT: Point‐voxel transformer for point cloud learning

Abstract

The recently developed pure transformer architectures have attained promising accuracy on point cloud learning benchmarks compared to convolutional neural networks. However, existing point cloud Transformers are computationally expensive because they waste a significant amount of time on structuring irregular data. To solve this shortcoming, we present the Sparse Window Attention module to gather coarse‐grained local features from nonempty voxels. The module not only bypasses the expensive irregular data structuring and invalid empty voxel computation, but also obtains linear computational complexity with respect to voxel resolution. Meanwhile, we leverage two different self‐attention variants to gather fine‐grained features about the global shape according to different scale of point clouds. Finally, we construct our neural architecture called point‐voxel transformer (PVT), which integrates these modules into a joint framework for point cloud learning. Compared with previous transformer‐based and attention‐based models, our method attains a top accuracy of 94.1% on the classification benchmark and 10-0.25em× $10\times $ inference speedup on average. Extensive experiments also validate the effectiveness of PVT on semantic segmentation benchmarks. Our code and pretrained model are avaliable at https://github.com/HaochengWan/PVT.

Semantic segmentation of point clouds of ancient buildings based on weak supervision

“…This will consume huge labor and time costs. In recent years, fully supervised networks are mainly studied on feature extractors [21,22,23,24,25,26,27]. Complex feature extractors cause their networks to require a large amount of computational resources.…”

ExpPoint-MAE: Better Interpretability and Performance for Self-Supervised Point Cloud Transformers

“…The feature maps of these layers are concatenated, and global features are extracted through mean and max pooling. In PVT [31], the authors create specialized attention modules for both points and voxels, utilizing the complementary information they contain to extract better features.…”

Rapid extraction of pavement aggregate gradation based on point clouds using deep learning networks

“…Another processing method of point clouds is voxelization. As two mainstream data forms representing three-dimensional space, the biggest difference between them is whether their points are orderly [31]. The point order exchange of the point cloud does not affect the point cloud shape, but the voxel is a kind of filling after representational space gridding, which means the voxels are ordered.…”