Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Tang, Haotian; Liu, Zhijian; Zhao, Shengyu; Lin, Yujun; Lin, Ji; Wang, Hanrui; Han, Song

doi:10.1007/978-3-030-58604-1_41

Cited by 420 publications

(273 citation statements)

References 48 publications

Supporting

Mentioning

220

Contrasting

Order By: Relevance

“…In the last few years, hybrid models where a mixture of voxel-based, projection-based and point-wise operations are used to learn features, have been investigated in order to take advantage of different representations of point clouds. For example, PVCNN [40] and SPVConv [41] use a point-voxel fusion scheme, in which voxels generate large-scale features, while point-wise MLPs are responsible for preserving fine geometrical features. FusionNet [42] also works on the point-voxel interaction.…”

Section: ) Deep Learning Based On Regular Representationsmentioning

confidence: 99%

A Comparison of Deep Learning Methods for Airborne Lidar Point Clouds Classification

Kähler

Pfeifer

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The success achieved by deep learning techniques in image labelling has triggered a growing interest in applying deep learning for 3D point cloud classification. To provide better insights into different deep learning architectures and their applications to ALS point cloud classification, this paper presents a comprehensive comparison among three state-of-theart deep learning networks: PointNet++, SparseCNN and KPConv, on two different ALS datasets. The performances of these three deep learning networks are compared w.r.t. classification accuracy, computation time, generalization ability as well as the sensitivity to the choices of hyper-parameters. Overall, we observed that PointNet++, SparseCNN and KPConv all outperform Random Forest on the classification results. Moreover, SparseCNN leads to a slightly better classification result compared to PointNet++ and KPConv, while requiring less computation time and memory. At the same time, it shows a better ability to generalize and is less impacted by the different choices of hyper-parameters.

show abstract

Section: ) Deep Learning Based On Regular Representationsmentioning

confidence: 99%

A Comparison of Deep Learning Methods for Airborne Lidar Point Clouds Classification

Kähler

Pfeifer

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Dynamic-OFA is a general approach for building dynamic DNNs, and the backbone network could be any super-networks trained by the OFA training pipeline. Our future work will investigate other applications such as network for IoT devices [12], transformers for natural language processing (NLP) tasks [18], generative adversarial networks (GANs) [11], 3D DNNs [17], etc.…”

Section: Discussionmentioning

confidence: 99%

Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms

Lou

Lei

Sabet

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Mobile and embedded platforms are increasingly required to efficiently execute computationally demanding DNNs across heterogeneous processing elements. At runtime, the available hardware resources to DNNs can vary considerably due to other concurrently running applications. The performance requirements of the applications could also change under different scenarios. To achieve the desired performance, dynamic DNNs have been proposed in which the number of channels/layers can be scaled in real time to meet different requirements under varying resource constraints. However, the training process of such dynamic DNNs can be costly, since platform-aware models of different deployment scenarios must be retrained to become dynamic. This paper proposes Dynamic-OFA, a novel dynamic DNN approach for state-of-the-art platform-aware NAS models (i.e. Once-for-all network (OFA)). Dynamic-OFA pre-samples a family of sub-networks from a static OFA backbone model, and contains a runtime manager to choose different sub-networks under different runtime environments. As such, Dynamic-OFA does not need the traditional dynamic DNN training pipeline. Compared to the state-of-the-art, our experimental results using ImageNet on a Jetson Xavier NX show that the approach is up to 3.5x (CPU), 2.4x (GPU) faster for similar Top-1 accuracy, or 3.8% (CPU), 5.1% (GPU) higher accuracy at similar latency.

show abstract

“…Graham et al [21] proposed sparse convolution to speed up volume convolution by keeping active sparseness and skipping calculations in inactive regions. SPVNAS [22] presents 3D Neural Architecture Search to efficiently and effectively search the optimal network architecture over this diverse design space.…”

Section: Structured Road Segmentation Methodsmentioning

confidence: 99%

“…However, the voxel-based representation requires very high resolution in order to not lose information, which will cause huge computational overhead. Therefore, this paper adopts the point-voxel [22] idea to design a substructure to extract features. Simultaneously, a cylinder is used instead of the voxel to solve voxel sparsity, which makes the point cloud distribution more legitimate with the Velodyne LiDAR data, effectively reducing the number of empty voxels and unnecessary calculations.…”

Section: Point-cylinder Substructurementioning

confidence: 99%

Unstructured Road Segmentation Based on Road Boundary Enhancement Point-Cylinder Network Using LiDAR Sensor

Zhu

Xu³

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

The segmentation of unstructured roads, a key technology in self-driving technology, remains a challenging problem. At present, most unstructured road segmentation algorithms are based on cameras or use LiDAR for projection, which has considerable limitations that the camera will fail at night, and the projection method will lose one-dimensional information. Therefore, this paper proposes a road boundary enhancement Point-Cylinder Network, called BE-PCFCN, which uses Point-Cylinder in order to extract point cloud features directly and integrates the road enhancement module to achieve accurate unstructured road segmentation. Firstly, we use the improved RANSAC-Boundary algorithm to calculate the rough road boundary point set, training in the same parameters with the original point cloud as a submodule. The whole network adopts the encoder and decoder structure, using Point-Cylinder as the basic module, while considering the data locality and the algorithm complexity. Subsequently, we made an unstructured road data set for training and compared it with existing LiDAR(Light Detection And Ranging) semantic segmentation algorithms. Finally, the experiment verified the robustness of BE-PCFCN. The road intersection-over-union (IoU) was increased by 4% when compared with the best existing algorithm, reaching 95.6%. Even on unstructured roads with an extremely irregular shape, BE-PCFCN also currently has the best segmentation results.

show abstract

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Cited by 420 publications

References 48 publications

A Comparison of Deep Learning Methods for Airborne Lidar Point Clouds Classification

A Comparison of Deep Learning Methods for Airborne Lidar Point Clouds Classification

Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms

Unstructured Road Segmentation Based on Road Boundary Enhancement Point-Cylinder Network Using LiDAR Sensor

Contact Info

Product

Resources

About