PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection

Shi, Shaoshuai; Jiang, Li; Deng, Jiajun; Wang, Zhe; Guo, Chunjie; Shi, Jianping; Wang, Xiaogang; Li, Hongsheng

doi:10.48550/arxiv.2102.00463

Cited by 37 publications

(51 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can see that our AFDetV2-Lite significantly outperforms the previous state-of-the-art single-frame LiDARonly detectors. To be specific, our AFDetV2-Lite achieves 68.77 APH/L2 for the mean of all three classes, surpassing prior art (Shi et al 2021) by 3.9%.…”

Section: Methodsmentioning

confidence: 89%

“…To tackle this problem, Part-A 2 (Shi et al 2020c) designed an RoI-aware point cloud pooling operation, while LiDAR R-CNN (Li, Wang, and Wang 2021) devised a method with virtual point and boundary offset. Another point pooling method proposed by PV-RCNN (Shi et al 2021) summarizes learned point and voxel-wise feature volumes at multiple neural layers into a small set of keypoints, then the keypoint features are aggregated according to RoI-grid. Pyramid R-CNN (Mao et al 2021a) utilizes an RoI-grid pyramid to mitigate the sparsity problem.…”

Section: Two Stage/singe Stage Lidar Detectormentioning

confidence: 99%

“…Object detection from point clouds has become a practical solution to robotics vision, especially in autonomous driving applications. Like the detection methods on 2D images, the 3D detection methods can also be divided into two groups: single-stage (Ge et al 2020;Zhou and Tuzel 2018;Yan, Mao, and Li 2018;He et al 2020;Zheng et al 2021;Lang et al 2019;Bewley et al 2020;Fan et al 2021;Chen et al 2020a) and two-stage (Shi, Wang, and Li 2019;Yang et al 2019;Qi et al 2017;Shi et al 2020c;Li, Wang, and Wang 2021;Shi et al 2021;Deng et al 2021a;Yin, Zhou, and Krahenbuhl 2021a;Sun et al 2021), in terms of the model structure. The two-stage methods usually show better accuracy (Shi, Wang, and Li 2019;Shi et al 2021;Deng et al 2021a) in the classification confidence and the box regression, than the single-stage methods.…”

Section: Introductionmentioning

confidence: 99%

“…Like the detection methods on 2D images, the 3D detection methods can also be divided into two groups: single-stage (Ge et al 2020;Zhou and Tuzel 2018;Yan, Mao, and Li 2018;He et al 2020;Zheng et al 2021;Lang et al 2019;Bewley et al 2020;Fan et al 2021;Chen et al 2020a) and two-stage (Shi, Wang, and Li 2019;Yang et al 2019;Qi et al 2017;Shi et al 2020c;Li, Wang, and Wang 2021;Shi et al 2021;Deng et al 2021a;Yin, Zhou, and Krahenbuhl 2021a;Sun et al 2021), in terms of the model structure. The two-stage methods usually show better accuracy (Shi, Wang, and Li 2019;Shi et al 2021;Deng et al 2021a) in the classification confidence and the box regression, than the single-stage methods. In these methods, the first stage provides proposals of the bounding boxes, based on which the second stage pools feature and runs through a smaller network to classify it and refine the box regression.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

Hu¹,

Ding²,

Ge³

et al. 2021

Preprint

View full text Add to dashboard Cite

There have been two streams in the 3D detection from point clouds: single-stage methods and two-stage methods. While the former is more computationally efficient, the latter usually provides better detection accuracy. By carefully examining the two-stage approaches, we have found that if appropriately designed, the first stage can produce accurate box regression. In this scenario, the second stage mainly rescores the boxes such that the boxes with better localization get selected. From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements. This network, named AFDetV2, extends the previous work by incorporating a self-calibrated convolution block in the backbone, a keypoint auxiliary supervision, and an IoU prediction branch in the multi-task head. We take a simple product of the predicted IoU score with the classification heatmap to form the final classification confidence. The enhanced backbone strengthens the box localization capability, and the rescoring approach effectively joins the object presence confidence and the box regression accuracy. As a result, the detection accuracy is drastically boosted in the single-stage. To evaluate our approach, we have conducted extensive experiments on the Waymo Open Dataset and the nuScenes Dataset. We have observed that our AFDetV2 achieves the state-of-theart results on these two datasets, superior to all the prior arts, including both the single-stage and the two-stage 3D detectors. AFDetV2 won the 1 st place in the Real-Time 3D Detection of the Waymo Open Dataset Challenge 2021. In addition, a variant of our model AFDetV2-Base was entitled the "Most Efficient Model" by the Challenge Sponsor, showing a superior computational efficiency. To demonstrate the generality of this single-stage method, we have also applied it to the first stage of the two-stage networks. Without exception, the results show that with the strengthened backbone and the rescoring approach, the second stage refinement is no longer needed.

show abstract

Section: Methodsmentioning

confidence: 89%

Section: Two Stage/singe Stage Lidar Detectormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

Hu¹,

Ding²,

Ge³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The voxel-based methods with predefined anchors have faster processing speed and can generate proposals with a higher recall rate, while the point-based methods can provide precise location information. In recent years, some methods have been proposed to fuse the advantages of voxel and point processing at the same time [2,23,28,29]. PV-RCNN [28] is a representative work among them, and its performance has reached the state of the arts, but due to the introduction of complex point operations, it cannot meet the real-time requirements (10 fps).…”

Section: Introductionmentioning

confidence: 99%

Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection

Zhang¹,

Zheng²,

et al. 2021

Preprint

View full text Add to dashboard Cite

Unlike 2D object detection where all RoI features come from grid pixels, the RoI feature extraction of 3D point cloud object detection is more diverse. In this paper, we first compare and analyze the differences in structure and performance between the two state-of-the-art models PV-RCNN and Voxel-RCNN. Then, we find that the performance gap between the two models does not come from point information, but structural information. The voxel features contain more structural information because they do quantization instead of downsampling to point cloud so that they can contain basically the complete information of the whole point cloud. The stronger structural information in voxel features makes the detector have higher performance in our experiments even if the voxel features don't have accurate location information. Then, we propose that structural information is the key to 3D object detection. Based on the above conclusion, we propose a Self-Attention RoI Feature Extractor (SARFE) to enhance structural information of the feature extracted from 3D proposals. SARFE is a plug-and-play module that can be easily used on existing 3D detectors. Our SARFE is evaluated on both KITTI dataset and Waymo Open dataset. With the newly introduced SARFE, we improve the performance of the state-ofthe-art 3D detectors by a large margin in cyclist on KITTI dataset while keeping real-time capability. The code will be released at https://github.com/Poley97/SARFE.

show abstract

PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds

Leng¹,

Cheng²,

Caine

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Data augmentation is an important technique to improve data efficiency and save labeling cost for 3D detection in point clouds. Yet, existing augmentation policies have so far been designed to only utilize labeled data, which limits the data diversity. In this paper, we recognize that pseudo labeling and data augmentation are complementary, thus propose to leverage unlabeled data for data augmentation to enrich the training data. In particular, we design three novel pseudolabel based data augmentation policies (PseudoAugments) to fuse both labeled and pseudo-labeled scenes, including frames (PseudoFrame), objects (PseudoBBox), and background (PseudoBackground). PseudoAugments outperforms pseudo labeling by mitigating pseudo labeling errors and generating diverse fused training scenes. We demonstrate Pseu-doAugments generalize across point-based and voxel-based architectures, different model capacity and both KITTI and Waymo Open Dataset. To alleviate the cost of hyperparameter tuning and iterative pseudo labeling, we develop a population-based data augmentation framework for 3D detection, named AutoPseudoAugment. Unlike previous works that perform pseudo-labeling offline, our framework performs PseudoAugments and hyperparameter tuning in one shot to reduce computational cost. Experimental results on the large-scale Waymo Open Dataset show our method outperforms state-of-the-art auto data augmentation method (PPBA) and self-training method (pseudo labeling). In particular, Au-toPseudoAugment is about 3× and 2× data efficient on vehicle and pedestrian tasks compared to prior arts. Notably, AutoPseudoAugment nearly matches the full dataset training results, with just 10% of the labeled run segments on the vehicle detection task.

show abstract

PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection

Cited by 37 publications

References 76 publications

AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection

PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds

Contact Info

Product

Resources

About