Probabilistic two-stage detection

Koltun, Vladlen

doi:10.48550/arxiv.2103.07461

Cited by 44 publications

(74 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CenterNet [74] first detects 2D objects in images and predicts the corresponding 3D depth and bounding box attributes using center features. Despite rapid progress, monocular 3D object detectors still perform far behind the Lidar-based [73] and point cloud based 3D detector [66]. We show detection from the 2D detector in blue and detection from 3D detector in green.…”

Section: Related Workmentioning

confidence: 97%

Multimodal Virtual Point 3D Detection

Yin¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Lidar-based sensing drives current autonomous vehicles. Despite rapid progress, current Lidar sensors still lag two decades behind traditional color cameras in terms of resolution and cost. For autonomous driving, this means that large objects close to the sensors are easily visible, but far-away or small objects comprise only one measurement or two. This is an issue, especially when these objects turn out to be driving hazards. On the other hand, these same objects are clearly visible in onboard RGB sensors. In this work, we present an approach to seamlessly fuse RGB sensors into Lidar-based 3D recognition. Our approach takes a set of 2D detections to generate dense 3D virtual points to augment an otherwise sparse 3D point cloud. These virtual points naturally integrate into any standard Lidar-based 3D detectors along with regular Lidar measurements. The resulting multi-modal detector is simple and effective. Experimental results on the large-scale nuScenes dataset show that our framework improves a strong CenterPoint baseline by a significant 6.6 mAP, and outperforms competing fusion approaches. Code and more visualizations are available at https://tianweiy.github.io/mvp/.

show abstract

Section: Related Workmentioning

confidence: 97%

Multimodal Virtual Point 3D Detection

Yin¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, given image region proposals from RPN, region representation extracted from our visual encoder are matched to the embeddings of target object concepts, thereby predicting the most likely category. Inspired by [47,63], we fuse RPN objectness scores and category confidence scores by geometry mean. Empirically, we observe that RPN scores significant improve zero-shot inference.…”

Section: Visual-semantic Alignment For Regionsmentioning

confidence: 99%

RegionCLIP: Region-based Language-Image Pretraining

Zhong¹,

Yang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Contrastive language-image pretraining (CLIP) using image-text pairs has achieved impressive results on image classification in both zero-shot and transfer learning settings. However, we show that directly applying such models to recognize image regions for object detection leads to poor performance due to a domain shift: CLIP was trained to match an image as a whole to a text description, without capturing the fine-grained alignment between image regions and text spans. To mitigate this issue, we propose a new method called RegionCLIP that significantly extends CLIP to learn region-level visual representations, thus enabling fine-grained alignment between image regions and textual concepts. Our method leverages a CLIP model to match image regions with template captions, and then pretrains our model to align these region-text pairs in the feature space. When transferring our pretrained model to the open-vocabulary object detection task, our method outperforms the state of the art by 3.8 AP50 and 2.2 AP for novel categories on COCO and LVIS datasets, respectively. Further, the learned region representations support zero-shot inference for object detection, showing promising results on both COCO and LVIS datasets. Our code is available at https://github.com/microsoft/RegionCLIP.

show abstract

“…Traditional object detection pipelines mostly employ the sliding window strategy, running a classifier on all ROIs. Early neural network based methods also follow this way, say the two-stage detectors [9,12,13,31,48]: Candidate ROIs are generated in the first stage, then are further classified in the second stage. Some subsequent works further improve the accuracy by importing multi-stage detection [4,40], and [27] tries to build relationships between candidate ROIs with RNN.…”

Section: Related Workmentioning

confidence: 99%

“…About the relationship between one-stage and two-stage detectors, they're not always competing, but can also co-work together as a stronger detector. For instance, most of those one-stage detectors can be integrated into a two-stage detection framework like FasterRCNN [31], working as the Region Proposal Network [48].…”

Section: Related Workmentioning

confidence: 99%

“…The occurrence of convolutional neural networks has brought in revolutionary improvements in various object detection tasks [10,14,24,39]. Generally, two-stage/multistage detectors [4,9,15,31,48] can achieve higher accuracy, while one-stage detectors [1,16,23,26,30,36] take better accuracy-performance balance. Recently, other than achieving better state-of-the-art results and less inference * Equal contribution cost, some research attentions are also paid to simplify training and inference pipelines.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Confidence Propagation Cluster: Unleash Full Potential of Object Detectors

Shen¹,

Jiang²,

Zhao³

et al. 2021

Preprint

View full text Add to dashboard Cite

It's been a long history that most object detection methods obtain objects by using the non-maximum suppression (NMS) and its improved versions like Soft-NMS to remove redundant bounding boxes. We challenge those NMS-based methods from three aspects: 1) The bounding box with highest confidence value may not be the true positive having the biggest overlap with the ground-truth box. 2) Not only suppression is required for redundant boxes, but also confidence enhancement is needed for those true positives.3) Sorting candidate boxes by confidence values is not necessary so that full parallelism is achievable.In this paper, inspired by belief propagation (BP), we propose the Confidence Propagation Cluster (CP-Cluster) to replace NMS-based methods, which is fully parallelizable as well as better in accuracy. In CP-Cluster, we borrow the message passing mechanism from BP to penalize redundant boxes and enhance true positives simultaneously in an iterative way until convergence. We verified the effectiveness of CP-Cluster by applying it to various mainstream detectors such as FasterRCNN, SSD, FCOS, YOLOv3, YOLOv5, Centernet etc. Experiments on MS COCO show that our plug and play method, without retraining detectors, is able to steadily improve average mAP of all those state-of-theart models with a clear margin from 0.2 to 1.9 respectively when compared with NMS-based methods. Source code is available at https://github.com/shenyi0220/CP-Cluster.

show abstract

Probabilistic two-stage detection

Cited by 44 publications

References 41 publications

Multimodal Virtual Point 3D Detection

Multimodal Virtual Point 3D Detection

RegionCLIP: Region-based Language-Image Pretraining

Confidence Propagation Cluster: Unleash Full Potential of Object Detectors

Contact Info

Product

Resources

About