Center-based 3D Object Detection and Tracking

Abstract

Three-dimensional objects are commonly represented as 3D boxes in a pointcloud. This representation mimics the well-studied image-based 2D boundingbox detection, but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. We use a keypoint detector to find centers of objects, and simply regress to other attributes, including 3D size, 3D orientation, and velocity. In our centerbased framework, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. On the nuScenes dataset, our point-based representations perform 3-4 mAP higher than the box-based counterparts for 3D detection, and 6 AMOTA higher for 3D tracking. Our real-time model runs end-to-end 3D detection and tracking at 30 FPS with 54.2 AMOTA and 48.3 mAP while the best single model achieves 60.3 mAP for 3D detection, and 63.8 AMOTA for 3D tracking. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

Person-MinkUNet: 3D Person Detection with LiDAR Point Cloud

“…For baseline, we train a CenterPoint using the original training setup from [16], except we reduce the voxel size to (0.05m, 0.05m, 0.2m) and disable the cut-paste data augmentation.…”

“…Of the latter category, submanifold sparse convolution [4,2] has gained success on various benchmarks, including ScanNet [3] and Se-manticKITTI [1]. For the task of 3D object detection, however, the current state-of-the-art is established by Center-Point [16] with VoxelNet [17] backbone.…”

Real-time 3D Object Detection using Feature Map Flow

“…While center-based 3D object detection and tracking method [13] is designed based on CenterNet [4] to represent, detect, and track 3D objects as points. CenterPoint framework [13], first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, Center-Point refines these estimates using additional point features on the object.…”

“…We followed CenterPoint [13] method in our detection head implementation to generate heat map peak at the center location of the detected objects by producing a K-channel heat map M , one channel for each of K classes.…”

“…We first compared our detection approach results with the state-of-the-art center-based 3D object detection and tracking method [13] (our baseline). Our main 3D object detection results on nuScenes validation set are shown in Table 2 and on Waymo validation set are shown in Table 4.…”