PointPillars: Fast Encoders for Object Detection From Point Clouds

Abstract

Object detection in point clouds is an important aspect of many robotics applications such as autonomous driving. In this paper we consider the problem of encoding a point cloud into a format appropriate for a downstream detection pipeline. Recent literature suggests two types of encoders; fixed encoders tend to be fast but sacrifice accuracy, while encoders that are learned from data are more accurate, but slower. In this work we propose PointPillars, a novel encoder which utilizes PointNets to learn a representation of point clouds organized in vertical columns (pillars). While the encoded features can be used with any standard 2D convolutional detection architecture, we further propose a lean downstream network. Extensive experimentation shows that PointPillars outperforms previous encoders with respect to both speed and accuracy by a large margin. Despite only using lidar, our full detection pipeline significantly outperforms the state of the art, even among fusion methods, with respect to both the 3D and bird's eye view KITTI benchmarks. This detection performance is achieved while running at 62 Hz: a 2 -4 fold runtime improvement. A faster version of our method matches the state of the art at 105 Hz. These benchmarks suggest that PointPillars is an appropriate encoding for object detection in point clouds.

SARPNET: Shape attention regional proposal network for liDAR-based 3D object detection

“…SECOND [28] utilizes Sparse Convolution [6] to accelerate VoxelNet and improve performance. Based on SECOND, PointPillars [10] point cloud. However, point number in each voxel is limited, leading to information loss.…”

“…HSV is a way of voxelization that assigns points to evenly spaced grid of voxels. The assignment phase in prior methods, like VoxelNet [35] and PointPillar [10], is accompanied by buffer allocation with a fixed size. Points will be dropped when the buffer capacity for a voxel is exceeded, causing randomness and information loss.…”

“…Instead, we maintain only the voxel index of each point. Assume that the point cloud is discretized [10]. The point-wise index c i for target voxel scale size s is recorded as a cursor to which voxel it lies in, given as:…”

“…Methods should be at least 20Hz since onboard application should cover 360 degree rather than KITTI annotation at limited 90 degree. Drawn methods are FP: F-PointNet [20], AF: AVOD-FPN [9], M: MMF [13], I: IPOD [31], FC: F-ConvNet [26], S: STD [32], PR: PointRCNN [22], FPR: Fast Point R-CNN [2], SE: SECOND [28], PP: PointPillars [10], PI: PIXOR++ [29] and O: our HVNet. For PointPillars we use their runtime on PyTorch for a fair comparison.…”

“…In order to handle these problems and utilize the advantages of LiDAR data, many approaches have been proposed recently (see Fig 1 caption). A noticeable line of methods combine 2D/3D voxelization with PointNet feature extractor in each voxel [10,28,35], which we call voxel feature encoding (VFE) methods. Using learnt point cloud features and 2D pseudo image feature map, these methods achieve both good accuracy and high inference speed.…”