AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

Abstract

There have been two streams in the 3D detection from point clouds: single-stage methods and two-stage methods. While the former is more computationally efficient, the latter usually provides better detection accuracy. By carefully examining the two-stage approaches, we have found that if appropriately designed, the first stage can produce accurate box regression. In this scenario, the second stage mainly rescores the boxes such that the boxes with better localization get selected. From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements. This network, named AFDetV2, extends the previous work by incorporating a self-calibrated convolution block in the backbone, a keypoint auxiliary supervision, and an IoU prediction branch in the multi-task head. We take a simple product of the predicted IoU score with the classification heatmap to form the final classification confidence. The enhanced backbone strengthens the box localization capability, and the rescoring approach effectively joins the object presence confidence and the box regression accuracy. As a result, the detection accuracy is drastically boosted in the single-stage. To evaluate our approach, we have conducted extensive experiments on the Waymo Open Dataset and the nuScenes Dataset. We have observed that our AFDetV2 achieves the state-of-theart results on these two datasets, superior to all the prior arts, including both the single-stage and the two-stage 3D detectors. AFDetV2 won the 1 st place in the Real-Time 3D Detection of the Waymo Open Dataset Challenge 2021. In addition, a variant of our model AFDetV2-Base was entitled the "Most Efficient Model" by the Challenge Sponsor, showing a superior computational efficiency. To demonstrate the generality of this single-stage method, we have also applied it to the first stage of the two-stage networks. Without exception, the results show that with the strengthened backbone and the rescoring approach, the second stage refinement is no longer needed.

DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection

“…According to the test results in Table 2, DeepFusion achieves the best results on Waymo Challenge Leaderboard demonstrating the effectiveness of our approach. For example, DeepFusion-Ens achieves the best results on the Waymo Challenge Leaderboard; DeepFusion improves 2.42 APH/L2 compared with previous state-of-theart single-model method, AFDetV2 [12].…”

“…Our very best model concatenates 5 frames, and with dropframe probabilities 0.5 during training. Besides, we also apply model ensemble and Test-Time Augmentation (TTA) by weighted box fusion (WBF) [12]. For TTA, we use yaw rotation, and global scaling.…”

Super Sparse 3D Object Detection

“…Based on this paradigm, many studies [15], [16], [17] improve the performance from different perspectives. Another hot trend aims to enhance the sparse backbone by transformer architecture [18], such as [4], [19], [20], [21], [22], [23], [24].…”

HOPE: Hierarchical Spatial-temporal Network for Occupancy Flow Prediction

“…For both models, we use Stochastic Weights Averaging (SWA) [12,13] to further enhance the training. We train each model for one additional epoch using decreased learning rate and greedily average the weights.…”