KDA3D: Key-Point Densification and Multi-Attention Guidance for 3D Object Detection

Abstract

In this paper, we propose a novel 3D object detector KDA3D, which achieves high-precision and robust classification, segmentation, and localization with the help of key-point densification and multi-attention guidance. The proposed end-to-end neural network architecture takes LIDAR point clouds as the main inputs that can be optionally complemented by RGB images. It consists of three parts: part-1 segments 3D foreground points and generates reliable proposals; part-2 (optional) enhances point cloud density and reconstructs the more compact full-point feature map; part-3 refines 3D bounding boxes and adds semantic segmentation as extra supervision. Our designed lightweight point-wise and channel-wise attention modules can adaptively strengthen the “skeleton” and “distinctiveness” point-features to help feature learning networks capture more representative or finer patterns. The proposed key-point densification component can generate pseudo-point clouds containing target information from monocular images through the distance preference strategy and K-means clustering so as to balance the density distribution and enrich sparse features. Extensive experiments on the KITTI and nuScenes 3D object detection benchmarks show that our KDA3D produces state-of-the-art results while running in near real-time with a low memory footprint.

3D Recognition Based on Sensor Modalities for Robotic Systems: A Survey

“…The results for car validation and test set are given in Tables 21 and 22 respectively while for pedestrian and cyclist categories on validation set are listed in Table 23. Wen et al [77] illustrates that the proposed model [77] competes with [199,201] in comprehensive performance. For the cyclist class, it outperforms the [201] while in the car class, it is 2× faster than [201].…”

“…Wen et al [77] illustrates that the proposed model [77] competes with [199,201] in comprehensive performance. For the cyclist class, it outperforms the [201] while in the car class, it is 2× faster than [201].…”

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

“…As for the LiDAR branch mentioned above, the point cloud can be used in the form of 3D points with reflectance, voxelized tensor, front-view/ range-view/ bird's eye view, as well as pseudo-point clouds. Though all these data have different intrinsic characteristics, which are highly associated with the latter LiDAR backbone, most of these data are produced with a rule-based procession except pseudo-point clouds [79]. Besides, all these data representations of Li-DAR can be visualized straightforward because the data in this stage still have interpretability compared to the embedding in feature space.…”

“…[54] transforms 3D lidar point clouds into the 2D image and fuses feature-level representation in the image branch leveraging mature CNN techniques to achieve better performance. [87] fused raw RGB pixel with the voxelized tensor while [79] directly combines pseudo-point clouds generated from the image branch and raw point clouds from the LiDAR branch together to accomplish object detection tasks.…”

Semantic Segmentation of 3D Point Cloud Based on Spatial Eight-Quadrant Kernel Convolution

“…Since the semantic understanding and analysis of a 3D point cloud is the basis for realizing scene understanding [1,2], the application of semantic segmentation of 3D point cloud has been more and more extensive in recent years [3][4][5], such as augmented/virtual reality [6] and intelligent robot [7]. Moreover, in the field of self-driving, the accurate perception of the environment based on LIDAR point cloud data is the key to realize information decision-making and driving safely in the complex dynamic environment.…”