AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

Abstract

There have been two streams in the 3D detection from point clouds: single-stage methods and two-stage methods. While the former is more computationally efficient, the latter usually provides better detection accuracy. By carefully examining the two-stage approaches, we have found that if appropriately designed, the first stage can produce accurate box regression. In this scenario, the second stage mainly rescores the boxes such that the boxes with better localization get selected. From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements. This network, named AFDetV2, extends the previous work by incorporating a self-calibrated convolution block in the backbone, a keypoint auxiliary supervision, and an IoU prediction branch in the multi-task head. We take a simple product of the predicted IoU score with the classification heatmap to form the final classification confidence. The enhanced backbone strengthens the box localization capability, and the rescoring approach effectively joins the object presence confidence and the box regression accuracy. As a result, the detection accuracy is drastically boosted in the single-stage. To evaluate our approach, we have conducted extensive experiments on the Waymo Open Dataset and the nuScenes Dataset. We have observed that our AFDetV2 achieves the state-of-the-art results on these two datasets, superior to all the prior arts, including both the single-stage and the two-stage 3D detectors. AFDetV2 won the 1st place in the Real-Time 3D Detection of the Waymo Open Dataset Challenge 2021. In addition, a variant of our model AFDetV2-Base was entitled the "Most Efficient Model" by the Challenge Sponsor, showing a superior computational efficiency. To demonstrate the generality of this single-stage method, we have also applied it to the first stage of the two-stage networks. Without exception, the results show that with the strengthened backbone and the rescoring approach, the second stage refinement is no longer needed.

Unifying Voxel-based Representation with Transformer for 3D Object Detection

“…In Table 8, we present comparisons with leading methods on the nuScenes test set. For LiDAR-based method, UVTR-L surpasses AFDetV2 [47] with 1.2% NDS and attains 69.7% NDS. For camera-based manner, with single frame, UVTR-L2C outperforms PETR [10] with 1.8% NDS and 1.1% mAP.…”

DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets

“…Without loss of generality, we follow the framework of CenterPoint-Pillar [48] and append our DSVT before BEV backbone. Besides that, we also follow [16,21,31] that uses IoU-rectification scheme to incorporate the IoU information into confidence scores. Two-stage model.…”

“…We adopt the same learning rate scheme as [48]. During inference, following [16,31], we use class-specific NMS with the IoU threshold of 0.7, 0.6 and 0.55 for vehicle, pedestrian and cyclist, respectively. All inference times are profiled on the same workstation (single NVIDIA A100 GPU and AMD EPYC 7513 CPU) and environment (Ubuntu-18.04, PyTorch-1.10.2, CUDA-11.3).…”

Det6D: A Ground-Aware Full-Pose 3D Object Detector for Improving Terrain Robustness

“…Most voxel-based methods compressed the feature volume along the z-axis into a BEV feature map. And then, an anchor-based [2], [3], [5], [6], [17] or anchor-free [4], [7], [8] detection head is employed to predict boxes.…”

“…3D object detection as an upstream task of these applications has attracted more researchers' attention. Most of them can be divided into grid-based methods [1]- [8] and point-based methods [9]- [13]. These methods are based on the flat-world assumption that objects are located on flat ground.…”