VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention

Abstract

Detecting objects from LiDAR point clouds is of tremendous significance in autonomous driving. In spite of good progress, accurate and reliable 3D detection is yet to be achieved due to the sparsity and irregularity of LiDAR point clouds. Among existing strategies, multi-view methods have shown great promise by leveraging the more comprehensive information from both bird's eye view (BEV) and range view (RV). These multi-view methods either refine the proposals predicted from single view via fused features, or fuse the features without considering the global spatial context; their performance is limited consequently. In this paper, we propose to adaptively fuse multi-view features in a global spatial context via Dual Cross-VIew SpaTial Attention (VISTA). The proposed VISTA is a novel plug-and-play fusion module, wherein the multi-layer perceptron widely adopted in standard attention modules is replaced with a convolutional one. Thanks to the learned attention mechanism, VISTA can produce fused features of high quality for prediction of proposals. We decouple the classification and regression tasks in VISTA, and an additional constraint of attention variance is applied that enables the attention module to focus on specific targets instead of generic points. We conduct thorough experiments on the benchmarks of nuScenes and Waymo; results confirm the efficacy of our designs. At the time of submission, our method achieves 63.0% in overall mAP and 69.8% in NDS on the nuScenes benchmark, outperforming all published methods by up to 24% in safety-crucial categories such as cyclist. Code.

SaBi3d - A LiDAR Point Cloud Data Set of Car-to-Bicyle Overtaking Maneuvers

“…While reproducing the results of VISTA [37] on nuScenes [16], it was discovered that the detection performance of VISTA heavily relies on the class-balanced grouping and sampling (CBGS) preprocessing step [45]. Without the resampling step, poor results were achieved (Table 2).…”

“…As a benchmark algorithm, VISTA [37] was chosen since it scored highest on the nuScenes leaderboard at the time. It uses a voxelized point cloud representation, various convolution operations [38][39][40], projection into two different views and cross-view transformers to bring the projections back together [41].…”

“…It uses a voxelized point cloud representation, various convolution operations [38][39][40], projection into two different views and cross-view transformers to bring the projections back together [41]. The core object detector can be varied, but the Object as Hotspots detector [42] yielded the best results [37]. Therefore, the same setup was used for the SaBi3d benchmark algorithm.…”

“…Along with the data set itself, we provide 3d object detection benchmark results for comparison with future works using the data set. As a baseline, the VISTA [37] algorithm was chosen. It was hypothesized that the voxelized point cloud representation would minimize the effect of the different scanning pattern of the Livox Horizon Lidar sensors.…”