Deep Learning-Based Monocular 3D Object Detection with Refinement of Depth Information

Abstract

Recently, the research on monocular 3D target detection based on pseudo-LiDAR data has made some progress. In contrast to LiDAR-based algorithms, the robustness of pseudo-LiDAR methods is still inferior. After conducting in-depth experiments, we realized that the main limitations are due to the inaccuracy of the target position and the uncertainty in the depth distribution of the foreground target. These two problems arise from the inaccurate depth estimation. To deal with the aforementioned problems, we propose two innovative solutions. The first is a novel method based on joint image segmentation and geometric constraints, used to predict the target depth and provide the depth prediction confidence measure. The predicted target depth is fused with the overall depth of the scene and results in the optimal target position. For the second, we utilize the target scale, normalized with the Gaussian function, as a priori information. The uncertainty of depth distribution, which can be visualized as long-tail noise, is reduced. With the refined depth information, we convert the optimized depth map into the point cloud representation, called a pseudo-LiDAR point cloud. Finally, we input the pseudo-LiDAR point cloud to the LiDAR-based algorithm to detect the 3D target. We conducted extensive experiments on the challenging KITTI dataset. The results demonstrate that our proposed framework outperforms various state-of-the-art methods by more than 12.37% and 5.34% on the easy and hard settings of the KITTI validation subset, respectively. On the KITTI test set, our framework also outperformed state-of-the-art methods by 5.1% and 1.76% on the easy and hard settings, respectively.

Power Operation Violation Identification Method Based on Point Cloud Data Preprocessing and Deep Learning under the Architecture of IoT

“…In order to prove the performance of the proposed method, it is compared with the methods in reference [26] and reference [27] under the same experimental conditions. Te comparison results are shown in Table 2.…”

“…Reference [26] introduces deformable convolution into the point cloud target detection network to enhance the adaptability of the network to detection targets in diferent directions and shapes. Reference [27] proposed a target detection method based on improved GUPNET. Te experimental results show that the average accuracy of the proposed method is the highest, with an average accuracy of 0.919 in simple difcult scenes, 0.897 in medium difcult scenes, and 0.839 in difcult scenes; reference [27] has an average accuracy of 0.900 in simple difcult scenes, 0.856 in medium difcult scenes, and 0.802 in difcult scenes; the average accuracy of reference [26] is the lowest, with the average accuracy of only 0.896 in simple difcult scenes, 0.879 in medium difcult scenes, and 0.783 in difcult scenes.…”

“…Reference [26] introduces deformable convolution into the point cloud target detection network to enhance the adaptability of the network to detection targets in diferent directions and shapes. Aiming at the imprecision of target position and the uncertainty of foreground target depth distribution, reference [27] proposes a target detection method based on improved GUPNET. However, the above methods often have the problems of low recognition accuracy and large memory occupation.…”

Consistency of Implicit and Explicit Features Matters for Monocular 3D Object Detection

“…On the KITTI 3D object detection benchmark, our model ranks first in all difficulty levels among all of the previously monocular models by achieving an AP moderate score of 20.95%. Our model outperforms the previously best 3D intermidiate representation method [7] by a large margin of 3.15%. On the KITTI bird's eye view benchmark, our model achieves an AP moderate score of 28.50%, outperforming all of the previously published models.…”

YOLOv4Tiny: Bearing Angle Based Pose Estimation and Semantic Segmentation For 3D Object Detection From LiDAR Point Cloud & RGB-D Data

“…However, the point clouds generated from photogrammetry are limited with poor precision which affects the object detection accuracy [2]. To overcome the issue of point clouds, LiDAR (Light Detection and Ranging) sensor is introduced which directly acquires the point clouds from the object and shows promising results in terms of accuracy, and provides faster results [3]. However, LiDARis susceptible to environmental noise, sparsity, and lack of depth information.…”