Understanding Strengths and Weaknesses of Complementary Sensor Modalities in Early Fusion for Object Detection

Corral-Soto, Eduardo R.; Liu, Bingbing

doi:10.1109/iv47402.2020.9304558

Cited by 10 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance of LiDAR 3D object detectors for autonomous driving has improved significantly in the recent years, thanks mainly to the evolution of deep learning detection architectures, and to the emergence of public labeled LiDAR point cloud datasets such as Waymo [1], NuScenes [2], KITTI [3], and ONCE [4]. The evolution of such detection architectures includes methods that voxelize the point cloud and either employ 3D convolutions [5], or 2D convolutions [6], [7], methods that operate on projections of LiDAR points [8], [9], [10], methods that operate directly on 3D points [11], and more recently, methods that combine voxelization with point-level processing [12]. For a comprehensive chronological overview of 3D object detection algorithms using LiDAR data see [13] .…”

Section: Introductionmentioning

confidence: 99%

“…Problem. It is well known that 3D point density distribution accross distance ranges of LiDAR point clouds for autonomous driving have a highly non-uniform distribution [24], [10]. This non-uniformity is mainly caused by the fixed scanning pattern of the LiDAR sensor and by its limited beam resolutions, which results in large density discrepancies between objects located at different distance ranges from the sensor.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Domain Adaptation in LiDAR Semantic Segmentation via Hybrid Learning with Alternating Skip Connections

Corral-Soto

Rochan

et al. 2023

2023 IEEE Intelligent Vehicles Symposium (IV)

Self Cite

View full text Add to dashboard Cite

In recent years, much progress has been made in LiDAR-based 3D object detection mainly due to advances in detector architecture designs and availability of large-scale LiDAR datasets. Existing 3D object detectors tend to perform well on the point cloud regions closer to the LiDAR sensor as opposed to on regions that are farther away. In this paper, we investigate this problem from the data perspective instead of detector architecture design. We observe that there is a learning bias in detection models towards the dense objects near the sensor and show that the detection performance can be improved by simply manipulating the input point cloud density at different distance ranges without modifying the detector architecture and without data augmentation. We propose a modelfree point cloud density adjustment pre-processing mechanism that uses iterative MCMC optimization to estimate optimal parameters for altering the point density at different distance ranges. We conduct experiments using four state-of-the-art LiDAR 3D object detectors on two public LiDAR datasets, namely Waymo and ONCE. Our results demonstrate that our range-based point cloud density manipulation technique can improve the performance of the existing detectors, which in turn could potentially inspire future detector designs.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Domain Adaptation in LiDAR Semantic Segmentation via Hybrid Learning with Alternating Skip Connections

Corral-Soto

Rochan

et al. 2023

2023 IEEE Intelligent Vehicles Symposium (IV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…There exist a number of techniques in the literature to address this problem, including transfer learning and fine-tuning, where the network is first trained on the source dataset, and then further trained (i.e. refined) using the available labeled target frames [2], [3], [4], [5], [6], [7], [8]. On the other hand, there is research in domain adaptation that focuses on reducing the domain shift between two or more domains.…”

Section: Introduction and Prior Workmentioning

confidence: 99%

Domain Adaptation in LiDAR Semantic Segmentation via Alternating Skip Connections and Hybrid Learning

Corral-Soto¹,

Rochan²,

Y.³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we address the problem of training a LiDAR semantic segmentation network using a fully-labeled source dataset and a target dataset that only has a small number of labels. To this end, we develop a novel imageto-image translation engine, and couple it with a LiDAR semantic segmentation network, resulting in an integrated domain adaptation architecture we call HYLDA. To train the system end-to-end, we adopt a diverse set of learning paradigms, including 1) self-supervision on a simple auxiliary reconstruction task, 2) semi-supervised training using a few available labeled target domain frames, and 3) unsupervised training on the fake translated images generated by the imageto-image translation stage, together with the labeled frames from the source domain. In the latter case, the semantic segmentation network participates in the updating of the imageto-image translation engine. We demonstrate experimentally that HYLDA effectively addresses the challenging problem of improving generalization on validation data from the target domain when only a few target labeled frames are available for training. We perform an extensive evaluation where we compare HYLDA against strong baseline methods using two publicly available LiDAR semantic segmentation datasets.

show abstract

“…This technique allows both to improve the detection of obstacles in the driving environment via multisensor data fusion [11] and to enhance safety in autonomous vehicles navigation [16]. The main advantage is that data collected from different sensors may contain complementary information [6], and data collected from remote sensors can help in filling blind spots [17]. To achieve data fusion, techniques such as the Kalman Filter [14], Extended Kalman Filter [21], Split Covariance Intersection Filter [18] and others have been used.…”

Section: Introductionmentioning

confidence: 99%

Augmented Perception with Cooperative Roadside Vision Systems for Autonomous Driving in Complex Scenarios

Masi

Ieng

2021

2021 IEEE International Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

Performing autonomous driving in urban environments is a challenging task, especially when there is a reduced visibility of traffic participants in complex driving scenarios. For this reason, we investigate the advantages of cooperative perception systems to enhance on-board perception capabilities. In this paper, we present a cooperative roadside vision system for augmenting the embedded perception of an autonomous vehicle navigating in a complex urban scenario. In particular, we use an HD map to implement a map-aided tracking system that merges the information from both onboard and remote sensors. The road users detected by the on-board LiDAR are represented as bounding polygons that include the localization uncertainty whereas, for the camera, the detected bounding boxes are projected in the map frame using a geometric constrained optimization. We report experimental results using two experimental vehicles and a roadside camera in a real traffic scenario in a roundabout. These results quantify how the cooperative data fusion extends the field of view and how the accuracy of the pose estimation of perceived objects is improved.

show abstract

Understanding Strengths and Weaknesses of Complementary Sensor Modalities in Early Fusion for Object Detection

Cited by 10 publications

References 6 publications

Domain Adaptation in LiDAR Semantic Segmentation via Hybrid Learning with Alternating Skip Connections

Domain Adaptation in LiDAR Semantic Segmentation via Hybrid Learning with Alternating Skip Connections

Domain Adaptation in LiDAR Semantic Segmentation via Alternating Skip Connections and Hybrid Learning

Augmented Perception with Cooperative Roadside Vision Systems for Autonomous Driving in Complex Scenarios

Contact Info

Product

Resources

About