PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Abstract

3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields. In this paper, we propose the Point-Voxel Region based Convolution Neural Networks (PV-RCNNs) for accurate 3D detection from point clouds. First, we propose a novel 3D object detector, PV-RCNN-v1, which employs the voxel-to-keypoint scene encoding and keypoint-to-grid RoI feature abstraction two novel steps. These two steps deeply incorporate both 3D voxel CNN and PointNet-based set abstraction for learning discriminative point-cloud features. Second, we propose a more advanced framework, PV-RCNN-v2, for more efficient and accurate 3D detection. It consists of two major improvements, where the first one is the sectorized proposal-centric strategy for efficiently producing more representative and uniformly distributed keypoints, and the second one is the VectorPool aggregation to replace set abstraction for better aggregating local point-cloud features with much less resource consumption. With these two major modifications, our PV-RCNN-v2 runs more than twice as fast as the v1 version while still achieving better performance on the large-scale Waymo Open Dataset with 150m × 150m detection range. Extensive experiments demonstrate that our proposed PV-RCNNs significantly outperform previous state-of-the-art 3D detection methods on both the Waymo Open Dataset and the highly-competitive KITTI benchmark.

PSANet: Pyramid Splitting and Aggregation Network for 3D Object Detection in Point Cloud

“…(2) Taking SECOND [ 12 ] as the baseline and expanding it to a two-stage detector. For example, PartA2 [ 34 ] and PV-RCNN [ 35 ] use SECOND [ 12 ] as the first stage of the detector and refine the high-quality proposals in the second stage. This type of method has a complicated structure and requires longer training and inference time on the same GPU.…”

RoIFusion: 3D Object Detection From LiDAR and Vision

“…This method allows for building a lightweight neural network for 116 by proposing an efficient method, named sparse convolution 117 [34], to ignore the empty voxels. In order to achieve high detection performance but also 129 to reduce the computational costs, several works [36]- [39] 130 introduced two-stages neural networks for 3D object detection.…”

Cascaded Cross-Modality Fusion Network for 3D Object Detection

“…In PointNet++, two strategies of multi-scale combination and multi-resolution combination are used to ensure more accurate target feature extraction. PV-RCNN [ 26 ] proposes a novel 3D object detection framework that deeply integrates both a 3D voxel convolutional neural network and a PointNet-based set abstraction. SA-SSD [ 27 ] is a point-based method, which improves accuracy by deeply mining the geometric information of three-dimensional objects.…”