Monocular 3D Object Detection via Geometric Reasoning on Keypoints

Barabanau, Ivan; Artemov, Alexey; Burnaev, Evgeny; Murashkin, Vyacheslav

doi:10.48550/arxiv.1905.05618

Cited by 16 publications

(22 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of 2D detectors as a starting point for 3D computation recently has become a standard approach [24,25]. Other works also explore advances in differentiable rendering [26] or 3D keypoint detection [27,28,1] to enable state-of-the-art 3D object detection performance. All these methods operate in a monocular setting, and extensions to multiple cameras are done by independently processing each frame before merging the outputs in a post-processing stage.…”

Section: Related Workmentioning

confidence: 99%

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

Wang¹,

Guizilini²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a framework for multi-camera 3D object detection. In contrast to existing works, which estimate 3D bounding boxes directly from monocular images or use depth prediction networks to generate input for 3D object detection from 2D information, our method manipulates predictions directly in 3D space. Our architecture extracts 2D features from multiple camera images and then uses a sparse set of 3D object queries to index into these 2D features, linking 3D positions to multi-view images using camera transformation matrices. Finally, our model makes a bounding box prediction per object query, using a set-to-set loss to measure the discrepancy between the ground-truth and the prediction. This top-down approach outperforms its bottom-up counterpart in which object bounding box prediction follows per-pixel depth estimation, since it does not suffer from the compounding error introduced by a depth prediction model. Moreover, our method does not require post-processing such as non-maximum suppression, dramatically improving inference speed. We achieve state-of-the-art performance on the nuScenes autonomous driving benchmark. *: Equal contribution. ¶: Co-advise on the project.

show abstract

Section: Related Workmentioning

confidence: 99%

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

Wang¹,

Guizilini²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Except for these methods, many methods tried to introduce geometry projection to infer depth [1,2,6,20]. Ivan et al [2] combined the keypoint method and the projection to do geometry reasoning. Decoupled3D [6] used lengths of bounding box edges to project and get the inferred depth.…”

Section: Related Workmentioning

confidence: 99%

Geometry Uncertainty Projection Network for Monocular 3D Object Detection

Yang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Researchers [64,65,66] first predict on a few discrete bins and then regress on the offset. There are also works designed to utilize a large number of bins or more exotic techniques: 12 non-overlapping equal bins [67], 72 non-overlapping bins [68], mean-shift after binning twice [36].…”

Section: N-bin and Affinity Representationsmentioning

confidence: 99%

SoK: Vehicle Orientation Representations for Deep Rotation Estimation

Tu¹,

Peng²,

Leung³

et al. 2021

Preprint

View full text Add to dashboard Cite

In recent years, there is an influx of deep learning models for 3D vehicle object detection. However, little attention was paid to orientation prediction. Existing research work proposed various vehicle orientation representation methods for deep learning, however a holistic, systematic review has not been conducted. Through our experiments, we categorize and compare the accuracy performance of various existing orientation representations using the KITTI 3D object detection dataset [1], and propose a new form of orientation representation: Tricosine. Among these, the 2D Cartesian-based representation [cos(θ), sin(θ)], or Single Bin, achieves the highest accuracy, with additional channeled inputs (positional encoding and depth map) not boosting prediction performance. Our code is published on GitHub: https://github.com/umd-fire-coml/KITTI-orientationlearning

show abstract

Monocular 3D Object Detection via Geometric Reasoning on Keypoints

Cited by 16 publications

References 0 publications

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

Geometry Uncertainty Projection Network for Monocular 3D Object Detection

SoK: Vehicle Orientation Representations for Deep Rotation Estimation

Contact Info

Product

Resources

About