Rethinking Pseudo-LiDAR Representation

Ma, Xinzhu; Liu, Shinan; Xia, Zhiyi; Zhang, Hongwen; Zeng, Xingyu; Ouyang, Wanli

doi:10.48550/arxiv.2008.04582

Cited by 9 publications

(24 citation statements)

References 37 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, monocular 3D object detection becomes a very popular area of research and develops quickly in recent years. Roughly speaking, monocular methods can be categorized into into image-only-based methods [18,23,4,1,12] and depth-map-based methods [30,31,16,5,15] according to input representations. Many previous works take advantage of prior knowledge and geometry constraints to help the estimating of complicated 3D parameters.…”

Section: Monocular 3d Object Detectionmentioning

confidence: 99%

Lidar Point Cloud Guided Monocular 3D Object Detection

Liang

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Monocular 3D object detection is drawing increasing attention from the community as it enables cars to perceive the world in 3D with a single camera. However, monocular 3D detection currently struggles with extremely lower detection rates compared to LiDAR-based methods, limiting its applications. The poor accuracy is mainly caused by the absence of accurate depth cues due to the ill-posed nature of monocular imagery. LiDAR point clouds, which provide accurate depth measurement, can offer beneficial information for the training of monocular methods. Prior works only use LiDAR point clouds to train a depth estimator. This implicit way does not fully utilize LiDAR point clouds, consequently leading to suboptimal performances. To effectively take advantage of LiDAR point clouds, in this paper we propose a general, simple yet effective framework for monocular methods. Specifically, we use LiDAR point clouds to directly guide the training of monocular 3D detectors, allowing them to learn desired objectives meanwhile eliminating the extra annotation cost. Thanks to the general design, our method can be plugged into any monocular 3D detection method, significantly boosting the performance. In conclusion, we take the first place on KITTI monocular 3D detection benchmark and increase the BEV/3D AP from 11.88/8.65 to 22.06/16.80 on the hard setting for the prior state-of-the-art method. The code will be made publicly available soon.

show abstract

Section: Monocular 3d Object Detectionmentioning

confidence: 99%

Lidar Point Cloud Guided Monocular 3D Object Detection

Liang

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, pseudo-LiDAR based methods ignore the significant gap between the transformed noisy point cloud and the accurate LiDAR point cloud. Furthermore, PatchNet [22] points out that the effectiveness of pseudo-LiDAR comes from the coordinates Orientation Estimation: Many approaches use the entire image directly for orientation estimation, such as [29,34,1,8]. The global semantics extracted from the region outside the object may interfere or even overwhelm the local semantic that matters for orientation estimation.…”

Section: Related Workmentioning

confidence: 99%

“…As shown in Fig. 1, stage one contains 2D detection and depth map estimation, producing 2D boxes and the image-based depth map by two off-the-shelf networks [27,16], as a common practice [36,37,23,22]. Our method mainly focuses on stage two that comprises object 3D center location and dimension & orientation prediction.…”

Section: Overviewmentioning

confidence: 99%

“…AP BEV /AP 3D (IoU=0. [36,8,22] are retrained using our proposed depth dataset split. To include more methods in comparison, all the methods are evaluated with metric AP | R11 .…”

Section: Approachmentioning

confidence: 99%

“…This depth dataset split contains data from non-overlapping sequences and fixes the data leakage issue. Thus we re-train and re-evaluate depth-based methods [36,8,22] using the official publicly available code.…”

Section: Dataset and Metricsmentioning

confidence: 99%

See 2 more Smart Citations

OCM3D: Object-Centric Monocular 3D Object Detection

Liang¹,

Liu²,

Yan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Image-only and pseudo-LiDAR representations are commonly used for monocular 3D object detection. However, methods based on them have shortcomings of either not well capturing the spatial relationships in neighbored image pixels or being hard to handle the noisy nature of the monocular pseudo-LiDAR point cloud. To overcome these issues, in this paper we propose a novel object-centric voxel representation tailored for monocular 3D object detection. Specifically, voxels are built on each object proposal, and their sizes are adaptively determined by the 3D spatial distribution of the points, allowing the noisy point cloud to be organized effectively within a voxel grid. This representation is proved to be able to locate the object in 3D space accurately. Furthermore, prior works would like to estimate the orientation via deep features extracted from an entire image or a noisy point cloud. By contrast, we argue that the local RoI information from the object image patch alone with a proper resizing scheme is a better input as it provides complete semantic clues meanwhile excludes irrelevant interferences. Besides, we decompose the confidence mechanism in monocular 3D object detection by considering the relationship between 3D objects and the associated 2D boxes. Evaluated on KITTI, our method outperforms state-of-the-art methods by a large margin. The code will be made publicly available soon.

show abstract

Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking

Jing

Rong

Kretzschmar

et al. 2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Monocular image-based 3D perception has become an active research area in recent years owing to its applications in autonomous driving. Approaches to monocular 3D perception including detection and tracking, however, often yield inferior performance when compared to LiDAR-based techniques. Through systematic analysis, we identified that perobject depth estimation accuracy is a major factor bounding the performance. Motivated by this observation, we propose a multilevel fusion method that combines different representations (RGB and pseudo-LiDAR) and temporal information across multiple frames for objects (tracklets) to enhance per-object depth estimation. Our proposed fusion method achieves the state-of-the-art performance of per-object depth estimation on the Waymo Open Dataset, the KITTI detection dataset, and the KITTI MOT dataset. We further demonstrate that by simply replacing estimated depth with fusion-enhanced depth, we can achieve significant improvements in monocular 3D perception tasks, including detection and tracking. * Work done while at Waymo LLC.1 In this paper we follow the tracking-by-detection paradigm.

show abstract

Rethinking Pseudo-LiDAR Representation

Cited by 9 publications

References 37 publications

Lidar Point Cloud Guided Monocular 3D Object Detection

Lidar Point Cloud Guided Monocular 3D Object Detection

OCM3D: Object-Centric Monocular 3D Object Detection

Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking

Contact Info

Product

Resources

About