Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics

Manfio, Barbosa, Felipe; Osório, Fernando Santos

doi:10.48550/arxiv.2303.04302

Cited by 4 publications

(7 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These environments consist of rectangular paper boxes labeled as 1 and 2 , a sphere labeled as 3 , another rectangular paper box labeled as 4 , and a conical barrier with base labeled as 7 ; the two sensors obtained point clouds information with varying depth distances within their sensing range. For the sphere labeled as 6 and the cylindrical barrier labeled as 8 , as well as the frame-type obstacle environment in Figure 10b which included benches, chairs and guardrails, the 2D LiDAR either failed to scan or captured minimal feature information. The PF method utilized the minimum distance principle to correctly extract the maximum contour information of the obstacle from the perspective of the LiDAR and camera detection after removing the invalid point cloud.…”

Section: Slam Map Construction Based On Pf Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Advancing Simultaneous Localization and Mapping with Multi-Sensor Fusion and Point Cloud De-Distortion

Shao,

Zhao,

Chen

et al. 2023

Machines

View full text Add to dashboard Cite

This study addresses the challenges associated with incomplete or missing information in obstacle detection methods that employ a single sensor. Additionally, it tackles the issue of motion distortion in LiDAR point cloud data during synchronization and mapping in complex environments. The research introduces two significant contributions. Firstly, a novel obstacle detection method, named the point-map fusion (PMF) algorithm, was proposed. This method integrates point cloud data from the LiDAR, camera, and odometer, along with local grid maps. The PMF algorithm consists of two components: the point-fusion (PF) algorithm, which combines LiDAR point cloud data and camera laser-like point cloud data through a point cloud library (PCL) format conversion and concatenation, and selects the most proximate point cloud to the quadruped robot dog as the valid data; and the map-fusion (MF) algorithm, which incorporates local grid maps acquired using the Gmapping and OctoMap algorithms, leveraging Bayesian estimation theory. The local grid maps obtained by the Gmapping and OctoMap algorithms are denoted as map A and map B, respectively. This sophisticated methodology enables seamless map fusion, which significantly enhances the precision and reliability of the approach. Secondly, a motion distortion removal (MDR) method for LiDAR point cloud data based on odometer readings was proposed. The MDR method utilizes legged odometer data for linear data interpolation of the original distorted LiDAR point cloud data, facilitating the determination of the corresponding pose of the quadruped robot dog. Subsequently, the LiDAR point cloud data are then transformed to the quadruped robot dog coordinate system, efficiently mitigating motion distortion. Experimental results demonstrated that the proposed PMF algorithm achieved a 50% improvement in success rate compared to using only LiDAR or the PF algorithm in isolation, while the MDR algorithm enhanced mapping accuracy by 45.9% when motion distortion was taken into account. The effectiveness of the proposed methods was confirmed through rigorous experimentation.

show abstract

Section: Slam Map Construction Based On Pf Methodsmentioning

confidence: 99%

“…Barbosa [8] fused 2D LiDAR and camera data using a convolutional neural network for segmentation tasks, and improved the obstacle detection accuracy for driving vehicles under adverse weather and strong lighting conditions. Wang [9] proposed an obstacle detection method based on machine learning and improved VIDAR.…”

Section: Introductionmentioning

confidence: 99%

Advancing Simultaneous Localization and Mapping with Multi-Sensor Fusion and Point Cloud De-Distortion

Shao,

Zhao,

Chen

et al. 2023

Machines

View full text Add to dashboard Cite

show abstract

“…The other researcher enhanced radar features through temporal accumulation and sent them to the spatiotemporal encoder for radar feature extraction, while obtaining multi-scale image 2D features that adapt to various spatial scales through the image backbone and Neck model. The designed initial map transformer was then used to convert image features into BEV [3].…”

Section: Related Workmentioning

confidence: 99%

“…Contemporary multi-sensor fusion encompasses various forms, such as the fusion of LiDAR, cameras and inertial measurement units [1], the fusion of millimetre wave radar and cameras [2], [3], [4], and the fusion of automotive chassis…”

Section: Introductionmentioning

confidence: 99%

Projection Mapping Segmentation Block: A Fusion Approach of Pointcloud and Image for Multi-Objects Classification

Meng

et al. 2023

IEEE Access

View full text Add to dashboard Cite

There is an increasing trend towards multi-modal sensor fusion to improve the perception capability of autonomous vehicles, such as the fusion of light detection and ranging (LiDAR) pointcloud and camera image data. In this work, we proposed the early-fusion method of projection mapping segmentation blocks, which is a method to map spatial pointcloud segmentation blocks and image pixel segmentation blocks to each other according to the projection relationship. The local spatial association feature and the local semi-variance texture feature were additionally designed for feature extraction, and each pair of segmentation blocks is used as a processing unit for feature extraction. Finally, the classification task was completed by Support Vector Machines (SVM) algorithm. We compared the classification results of the single pointcloud without fusion and pointcloud+image fusion under two segmentation methods: boxbased and sector ring hexahedron-based. Our findings suggest that the data fusion classification method based on sector ring hexahedron segmentation not only improves classification accuracy but also reduces computational resources, yielding encouraging results.

show abstract

“…Barbosa and Osorio, in [21], study radar-based perception and radar-camera fusion, as applied to the navigation of an autonomous vehicle moving in unfavorable lighting and weather conditions.…”

Section: Introductionmentioning

confidence: 99%

Radar Perception of Multi-Object Collision Risk Neural Domains during Autonomous Driving

Lisowski

2024

Electronics

View full text Add to dashboard Cite

The analysis of the state of the literature in the field of methods of perception and control of the movement of autonomous vehicles shows the possibilities of improving them by using an artificial neural network to generate domains of prohibited maneuvers of passing objects, contributing to increasing the safety of autonomous driving in various real conditions of the surrounding environment. This article concerns radar perception, which involves receiving information about the movement of many autonomous objects, then identifying and assigning them a collision risk and preparing a maneuvering response. In the identification process, each object is assigned a domain generated by a previously trained neural network. The size of the domain is proportional to the risk of collisions and distance changes during autonomous driving. Then, an optimal trajectory is determined from among the possible safe paths, ensuring control in a minimum of time. The presented solution to the radar perception task was illustrated with a computer simulation of autonomous driving in a situation of passing many objects. The main achievements presented in this article are the synthesis of a radar perception algorithm mapping the neural domains of autonomous objects characterizing their collision risk and the assessment of the degree of radar perception on the example of multi-object autonomous driving simulation.

show abstract

Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics

Cited by 4 publications

References 71 publications

Advancing Simultaneous Localization and Mapping with Multi-Sensor Fusion and Point Cloud De-Distortion

Advancing Simultaneous Localization and Mapping with Multi-Sensor Fusion and Point Cloud De-Distortion

Projection Mapping Segmentation Block: A Fusion Approach of Pointcloud and Image for Multi-Objects Classification

Radar Perception of Multi-Object Collision Risk Neural Domains during Autonomous Driving

Contact Info

Product

Resources

About