Boosting Monocular 3D Object Detection With Object-Centric Auxiliary Depth Supervision

Kim, Youngseok; Kim, Sanmin; Sim, Sangmin; Choi, Jun Won; Kum, Dongsuk

doi:10.1109/tits.2022.3224082

Cited by 6 publications

(4 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Radars longer wavelength compared to other laser sensors enables them to be the only perception sensor, for which performance doesn't degrade with adverse weather conditions viz., rain/ snow/ dust etc. These characteristics is very well summarized by [5] in Fig. 3.…”

Section: B Choice Of Sensorsmentioning

confidence: 64%

“…3) Transformer based: This line of work typically utilizes transformers module viz., cross-attention to cross-attend features from different modalities and form a finer feature representation. A representative work in CRAFT [5] associates image proposals with radar point in the polar coordinate system to efficiently handle the discrepancy between the coordinate system and spatial properties. Then in second stage, they use consecutive cross-attention based feature fusion layers to share spatio-contextual information between camera and radar.…”

Section: Deep Fusionmentioning

confidence: 99%

“…Reference [5] has shown camera and radar's characteristics and how they complement each other. Cameras are not very good in generalizing in BEV predictions, as the input they receive is constrained by 2D pixels.…”

Section: Introduction Sae (Society Of Automotive Engineersmentioning

confidence: 99%

See 2 more Smart Citations

Vision-RADAR fusion for Robotics BEV Detections: A Survey

Singh¹

2023

Preprint

View full text Add to dashboard Cite

Due to the trending need of building autonomous robotic perception system, sensor fusion has attracted a lot of attention amongst researchers and engineers to make best use of cross-modality information. However, in order to build a robotic platform at scale we need to emphasize on autonomous robot platform bring-up cost as well. Cameras and radars, which inherently includes complementary perception information, has potential for developing autonomous robotic platform at scale. However, there is a limited work around radar fused with Vision, compared to LiDAR fused with vision work. In this paper, we tackle this gap with a survey on Vision-Radar fusion approaches for a BEV object detection system. First we go through the background information viz., object detection tasks, choice of sensors, sensor setup, benchmark datasets and evaluation metrics for a robotic perception system. Later, we cover per-modality (Camera and RADAR) data representation, then we go into detail about sensor fusion techniques based on sub-groups viz., early-fusion, deep-fusion, and late-fusion to easily understand the pros and cons of each method. Finally, we propose possible future trends for vision-radar fusion to enlighten future research.

show abstract

Section: B Choice Of Sensorsmentioning

confidence: 64%

Section: Deep Fusionmentioning

confidence: 99%

See 1 more Smart Citation

Vision-RADAR fusion for Robotics BEV Detections: A Survey

Singh¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The decoder incorporates the refined depth estimation log standard deviation log (σ) predicted by the secondary depth uncertainty head into the final 3D detection confidence p 3D following MonoDDE [34] and MonoPixel [77]. By doing so, the confidence of candidates with high depth estimation uncertainty can be attenuated, thus filtering out candidates that are poorly localized.…”

Section: E 3d Bounding Box Decodermentioning

confidence: 99%

ClusterFusion: Leveraging Radar Spatial Features for Radar-Camera 3D Object Detection in Autonomous Vehicles

Kurniawan,

Trilaksono

2023

IEEE Access

View full text Add to dashboard Cite

Thanks to the complementary nature of millimeter wave radar and camera, deep learningbased radar-camera 3D object detection methods may reliably produce accurate detections even in lowvisibility conditions. This makes them preferable to use in autonomous vehicles' perception systems, especially as the combined cost of both sensors is cheaper than the cost of a lidar. Recent radar-camera methods commonly perform feature-level fusion which often involves projecting the radar points onto the same plane as the image features and fusing the extracted features from both modalities. While performing fusion on the image plane is generally simpler and faster, projecting radar points onto the image plane flattens the depth dimension of the point cloud which might lead to information loss and makes extracting the spatial features of the point cloud harder. We proposed ClusterFusion, an architecture that leverages the local spatial features of the radar point cloud by clustering the point cloud and performing feature extraction directly on the point cloud clusters before projecting the features onto the image plane. ClusterFusion achieved the state-of-the-art performance among all radar-monocular camera methods on the test slice of the nuScenes dataset with 48.7% nuScenes detection score (NDS). We also investigated the performance of different radar feature extraction strategies on point cloud clusters: a handcrafted strategy, a learning-based strategy, and a combination of both, and found that the handcrafted strategy yielded the best performance. The main goal of this work is to explore the use of radar's local spatial and point-wise features by extracting them directly from radar point cloud clusters for a radar-monocular camera 3D object detection method that performs cross-modal feature fusion on the image plane.

show abstract

MonoLI: Precise Monocular 3-D Object Detection for Next-Generation Consumer Electronics for Autonomous Electric Vehicles

Gao,

Yu,

et al. 2024

IEEE Trans. Consumer Electron.

View full text Add to dashboard Cite

Boosting Monocular 3D Object Detection With Object-Centric Auxiliary Depth Supervision

Cited by 6 publications

References 51 publications

Vision-RADAR fusion for Robotics BEV Detections: A Survey

Vision-RADAR fusion for Robotics BEV Detections: A Survey

ClusterFusion: Leveraging Radar Spatial Features for Radar-Camera 3D Object Detection in Autonomous Vehicles

MonoLI: Precise Monocular 3-D Object Detection for Next-Generation Consumer Electronics for Autonomous Electric Vehicles

Contact Info

Product

Resources

About