Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

Meyer, Gregory P.; Charland, Jake; Hegde, Darshan; Laddha, Ankit; Vallespi-Gonzalez, Carlos

doi:10.1109/cvprw.2019.00162

Cited by 123 publications

(85 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposed method outperforms the previous methods that only utilize LiDAR data. Furthermore, by simply changing the loss function, we observe a similar gain in performance as adding an additional sensing modality (see LaserNet++ [11]). Fig.…”

Section: A Detection Evaluationmentioning

confidence: 72%

“…To accomplish this task, autonomous vehicles are equipped with various sensors including cameras and LiDARs. A wealth of deep learning based approaches have been proposed to perform 3D object detection using these sensors [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. Given the limited sensory information, it is unrealistic to expect any detector to flawlessly classify and localize every actor in all situations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving

Meyer

Laddha

Kee

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

332

231

View full text Add to dashboard Cite

The capability to detect objects is a core part of autonomous driving. Due to sensor noise and incomplete data, perfectly detecting and localizing every object is infeasible. Therefore, it is important for a detector to provide the amount of uncertainty in each prediction. Providing the autonomous system with reliable uncertainties enables the vehicle to react differently based on the level of uncertainty. Previous work has estimated the uncertainty in a detection by predicting a probability distribution over object bounding boxes. In this work, we propose a method to improve the ability to learn the probability distribution by considering the potential noise in the ground-truth labeled data. Our proposed approach improves not only the accuracy of the learned distribution but also the object detection performance.

show abstract

Section: A Detection Evaluationmentioning

confidence: 72%

Section: Introductionmentioning

confidence: 99%

LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving

Meyer

Laddha

Kee

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

332

231

View full text Add to dashboard Cite

show abstract

“…Methods such as [20,21] propose the idea of multi-sensor fusion networks [20,21] to increase the model accuracy, but despite high accuracy, these methods are computationally expensive. To tackle the sensor-fusion computational problem, this [22] proposed an earlyfusion method to fuse both camera and LiDAR with only one backbone, attaining a good balance between accuracy and efficiency.…”

Section: D Object Detectionmentioning

confidence: 99%

Transfer Learning Based Semantic Segmentation for 3D Object Detection from Point Cloud

Imad

Doukhi

Lee

2021

Sensors

View full text Add to dashboard Cite

Three-dimensional object detection utilizing LiDAR point cloud data is an indispensable part of autonomous driving perception systems. Point cloud-based 3D object detection has been a better replacement for higher accuracy than cameras during nighttime. However, most LiDAR-based 3D object methods work in a supervised manner, which means their state-of-the-art performance relies heavily on a large-scale and well-labeled dataset, while these annotated datasets could be expensive to obtain and only accessible in the limited scenario. Transfer learning is a promising approach to reduce the large-scale training datasets requirement, but existing transfer learning object detectors are primarily for 2D object detection rather than 3D. In this work, we utilize the 3D point cloud data more effectively by representing the birds-eye-view (BEV) scene and propose a transfer learning based point cloud semantic segmentation for 3D object detection. The proposed model minimizes the need for large-scale training datasets and consequently reduces the training time. First, a preprocessing stage filters the raw point cloud data to a BEV map within a specific field of view. Second, the transfer learning stage uses knowledge from the previously learned classification task (with more data for training) and generalizes the semantic segmentation-based 2D object detection task. Finally, 2D detection results from the BEV image have been back-projected into 3D in the postprocessing stage. We verify results on two datasets: the KITTI 3D object detection dataset and the Ouster LiDAR-64 dataset, thus demonstrating that the proposed method is highly competitive in terms of mean average precision (mAP up to 70%) while still running at more than 30 frames per second (FPS).

show abstract

“…There are various types of sensor fusion depending on where the data are projected. For studies using a bird’s eye-view (BEV), there is the fusion method of cameras and LiDAR by using the deep learning technique after calibration [ 1 ]. In many studies, fusion is performed by projecting the point cloud of LiDAR onto the image space.…”

Section: Introductionmentioning

confidence: 99%

Estimation of the Closest In-Path Vehicle by Low-Channel LiDAR and Camera Sensor Fusion for Autonomous Vehicles

Bae

Lee

Yang

et al. 2021

Sensors

View full text Add to dashboard Cite

In autonomous driving, using a variety of sensors to recognize preceding vehicles at middle and long distances is helpful for improving driving performance and developing various functions. However, if only LiDAR or cameras are used in the recognition stage, it is difficult to obtain the necessary data due to the limitations of each sensor. In this paper, we proposed a method of converting the vision-tracked data into bird’s eye-view (BEV) coordinates using an equation that projects LiDAR points onto an image and a method of fusion between LiDAR and vision-tracked data. Thus, the proposed method was effective through the results of detecting the closest in-path vehicle (CIPV) in various situations. In addition, even when experimenting with the EuroNCAP autonomous emergency braking (AEB) test protocol using the result of fusion, AEB performance was improved through improved cognitive performance than when using only LiDAR. In the experimental results, the performance of the proposed method was proven through actual vehicle tests in various scenarios. Consequently, it was convincing that the proposed sensor fusion method significantly improved the adaptive cruise control (ACC) function in autonomous maneuvering. We expect that this improvement in perception performance will contribute to improving the overall stability of ACC.

show abstract

Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

Cited by 123 publications

References 32 publications

LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving

LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving

Transfer Learning Based Semantic Segmentation for 3D Object Detection from Point Cloud

Estimation of the Closest In-Path Vehicle by Low-Channel LiDAR and Camera Sensor Fusion for Autonomous Vehicles

Contact Info

Product

Resources

About