Multi-View 3D Object Detection Network for Autonomous Driving

Chen, Xiaozhi; Wan, Ji; Li, Bo; Xia, Tian

doi:10.48550/arxiv.1611.07759

Cited by 35 publications

(28 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li [15] used 3D point cloud data and proposed to use 3D convolutions on a voxelized representation of point clouds. Chen et al [3] combined image and 3D point clouds with a fusion network. They exploited 2D convolutions in BEV, however, they used hand-crafted height features as input.…”

Section: D Object Detectionmentioning

confidence: 99%

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

Luo

Yang

Urtasun

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

571

469

View full text Add to dashboard Cite

In this paper we propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor. By jointly reasoning about these tasks, our holistic approach is more robust to occlusion as well as sparse data at range. Our approach performs 3D convolutions across space and time over a bird's eye view representation of the 3D world, which is very efficient in terms of both memory and computation. Our experiments on a new very large scale dataset captured in several north american cities, show that we can outperform the state-of-the-art by a large margin. Importantly, by sharing computation we can perform all tasks in as little as 30 ms.

show abstract

Section: D Object Detectionmentioning

confidence: 99%

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

Luo

Yang

Urtasun

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

571

469

View full text Add to dashboard Cite

show abstract

“…LiDARs have been widely used for 3D object detection and tracking in autonomous driving applications in recent years. The majority of LiDAR-based methods either use 3D voxels [12,35] or 2D projections [13,5,29,31] for point cloud representation. Voxel-based methods are usually slow as a result of the voxel grid's high dimensionality, and projection-based methods might suffer from large variances in object shapes and sizes depending on the projection plane.…”

Section: Single-modality Methodsmentioning

confidence: 99%

CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection

Nabati,

2020

Preprint

View full text Add to dashboard Cite

The perception system in autonomous vehicles is responsible for detecting and tracking the surrounding objects. This is usually done by taking advantage of several sensing modalities to increase robustness and accuracy, which makes sensor fusion a crucial part of the perception system. In this paper, we focus on the problem of radar and camera sensor fusion and propose a middle-fusion approach to exploit both radar and camera data for 3D object detection. Our approach, called CenterFusion, first uses a center point detection network to detect objects by identifying their center points on the image. It then solves the key data association problem using a novel frustum-based method to associate the radar detections to their corresponding object's center point. The associated radar detections are used to generate radar-based feature maps to complement the image features, and regress to object properties such as depth, rotation and velocity. We evaluate CenterFusion on the challenging nuScenes dataset, where it improves the overall nuScenes Detection Score (NDS) of the state-of-the-art camera-based algorithm by more than 12%. We further show that CenterFusion significantly improves the velocity estimation accuracy without using any additional temporal information. The code is available at https://github.com/mrnabati/CenterFusion.

show abstract

“…Sensory fusion approaches have been widely used in computer vision. LIDAR and camera are popular sensor sets employed in detection and tracking [21], [22], [23]. Other papers also exploit radar [24].…”

Section: Related Workmentioning

confidence: 99%

“…1) Detection θ det W (x): We exploit object proposals in order to reduce the search space over all possible detections. In particular, we employ the MV3D detector [22] to produce oriented 3D object proposals from LIDAR and RGB data (i.e., regions in 3D where there is a high probability that a vehicle is present). To make sure that the tracker produces accurate trajectories, we need a classifier that decides whether or not an object proposal is a true positive (i.e., actually represents a vehicle).…”

Section: Deep Scoring and Matchingmentioning

confidence: 99%

See 1 more Smart Citation

End-to-end Learning of Multi-sensor 3D Tracking by Detection

Frossard

Urtasun

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

106

View full text Add to dashboard Cite

In this paper we propose a novel approach to tracking by detection that can exploit both cameras as well as LIDAR data to produce very accurate 3D trajectories. Towards this goal, we formulate the problem as a linear program that can be solved exactly, and learn convolutional networks for detection as well as matching in an end-to-end manner. We evaluate our model in the challenging KITTI dataset and show very competitive results.

show abstract

Multi-View 3D Object Detection Network for Autonomous Driving

Cited by 35 publications

References 24 publications

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net

CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection

End-to-end Learning of Multi-sensor 3D Tracking by Detection

Contact Info

Product

Resources

About