Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations

Bhattacharyya, Prarthana; Czarnecki, Krzysztof

doi:10.48550/arxiv.2008.08766

Cited by 3 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in the table, our model ranks the 1 st place among all state-of-the-art methods for both 3D and BEV detections in all three difficulty levels. Also, the inference speed of our model ranks the 2 nd place among all methods, about 2.6 times faster than the latest best two-stage detector Deformable PV-RCNN [1]. In 3D detection, our one-stage model attains a significant improvement of 1.1 points on moderate AP compared with PV-RCNN [1] and Deformable PV-RCNN [23].…”

Section: Comparison With State-of-the-artsmentioning

confidence: 80%

“…For example, the series of works [24,4,25,23] focus on improving the region-proposal-aligned features for a better refinement with a second-stage network. Also, many methods [3,10,29,12,33,19] try to extract more discrimina- 1 On the date of CVPR deadline, i.e., Nov 16, 2020 Our SE-SSD attains top precisions on both 3D and BEV car detection in KITTI benchmark [6] with real-time speed (30.56 ms), clearly outperforming all state-of-the-art detectors. Please refer to Table 1 for a detailed comparison with more methods.…”

Section: Introductionmentioning

confidence: 95%

See 1 more Smart Citation

SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud

Wu¹,

Tang²,

Jiang³

2021

Preprint

View full text Add to dashboard Cite

We present Self-Ensembling Single-Stage object Detector (SE-SSD) for accurate and efficient 3D object detection in outdoor point clouds. Our key focus is on exploiting both soft and hard targets with our formulated constraints to jointly optimize the model, without introducing extra computation in the inference. Specifically, SE-SSD contains a pair of teacher and student SSDs, in which we design an effective IoU-based matching strategy to filter soft targets from the teacher and formulate a consistency loss to align student predictions with them. Also, to maximize the distilled knowledge for ensembling the teacher, we design a new augmentation scheme to produce shape-aware augmented samples to train the student, aiming to encourage it to infer complete object shapes. Lastly, to better exploit hard targets, we design an ODIoU loss to supervise the student with constraints on the predicted box centers and orientations. Our SE-SSD attains top performance compared with all prior published works. Also, it attains top precisions for car detection in the KITTI benchmark (ranked 1 st and 2 nd on the BEV and 3D leaderboards 1 , respectively) with an ultra-high inference speed. The code is available at https://github.com/Vegeta2020/SE-SSD.

show abstract

Section: Comparison With State-of-the-artsmentioning

confidence: 80%

Section: Introductionmentioning

confidence: 95%

SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud

Wu¹,

Tang²,

Jiang³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The voxel-based methods with predefined anchors have faster processing speed and can generate proposals with a higher recall rate, while the point-based methods can provide precise location information. In recent years, some methods have been proposed to fuse the advantages of voxel and point processing at the same time [2,23,28,29]. PV-RCNN [28] is a representative work among them, and its performance has reached the state of the arts, but due to the introduction of complex point operations, it cannot meet the real-time requirements (10 fps).…”

Section: Introductionmentioning

confidence: 99%

“…There are two main differences between them: (1) PV-RCNN has a point branch to get the point features and an extra loss for the foreground point segmentation task, while Voxel-RCNN doesn't have them. (2) The features used for RoI pooling are different. Voxel-RCNN pooling features from the 3D voxel backbone while PV-RCNN pooling features from weighted point features from the point branch.…”

Section: Introductionmentioning

confidence: 99%

“…The main contributions of this work can be summarized as (1) We indicate the essential difference between point-based and voxel-based RoI feature extractors is the tradeoff between location and structure information. Then we point out that compared to accurate position information, structural information is more important for 3D object detection; (2) We propose a plug-and-play module called self-attention RoI Feature Extractor (SARFE) to automatically adjust local features in a proposal to get features with stronger structure information; (3) Our proposed method SARFE achieves newly the-state-of-art performance on cyclist of KITTI dataset [8] while keeping real-time ability. SARFE also achieves state-of-the-art performance on the large-scale Waymo Open dataset [34].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection

Zhang¹,

Zheng²,

et al. 2021

Preprint

View full text Add to dashboard Cite

Unlike 2D object detection where all RoI features come from grid pixels, the RoI feature extraction of 3D point cloud object detection is more diverse. In this paper, we first compare and analyze the differences in structure and performance between the two state-of-the-art models PV-RCNN and Voxel-RCNN. Then, we find that the performance gap between the two models does not come from point information, but structural information. The voxel features contain more structural information because they do quantization instead of downsampling to point cloud so that they can contain basically the complete information of the whole point cloud. The stronger structural information in voxel features makes the detector have higher performance in our experiments even if the voxel features don't have accurate location information. Then, we propose that structural information is the key to 3D object detection. Based on the above conclusion, we propose a Self-Attention RoI Feature Extractor (SARFE) to enhance structural information of the feature extracted from 3D proposals. SARFE is a plug-and-play module that can be easily used on existing 3D detectors. Our SARFE is evaluated on both KITTI dataset and Waymo Open dataset. With the newly introduced SARFE, we improve the performance of the state-ofthe-art 3D detectors by a large margin in cyclist on KITTI dataset while keeping real-time capability. The code will be released at https://github.com/Poley97/SARFE.

show abstract

JPV-Net: Joint Point-Voxel Representations for Accurate 3D Object Detection

Song

Jiang

Yao

2022

AAAI

View full text Add to dashboard Cite

Voxel and point representations are widely applied in recent 3D object detection tasks from LiDAR point clouds. Voxel representations contribute to efficiently and rapidly locating objects, whereas point representations are capable of describing intra-object spatial relationship for detection refinement. In this work, we aim to exploit the strengths of both two representations, and present a novel two-stage detector, named Joint Point-Voxel Network (JPV-Net). Specifically, our framework is equipped with a Dual Encoders-Fusion Decoder, which consists of the dual encoders to extract voxel features of sketchy 3D scenes and point features rich in geometric context, respectively, and the Feature Propagation Fusion (FP-Fusion) decoder to attentively fuse them from coarse to fine. By making use of the advantages of these features, the refinement network can effectively eliminate false detection and provide better accuracy. Besides, to further develop the perception characteristics of voxel CNN and point backbone, we design two novel intersection-over-union (IoU) estimation modules for proposal generation and refinement, both of which can alleviate the misalignment between the localization and the classification confidence. Extensive experiments on the KITTI dataset and ONCE dataset demonstrate that our proposed JPV-Net outperforms other state-of-the-art methods with remarkable margins.

show abstract

Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations

Cited by 3 publications

References 0 publications

SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud

SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud

Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection

JPV-Net: Joint Point-Voxel Representations for Accurate 3D Object Detection

Contact Info

Product

Resources

About