Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS

Abstract

The perception module plays an important role in vehicles equipped with advanced driver-assistance systems (ADAS). This paper presents a multi-sensor data fusion system based on the polarization color stereo camera and the forward-looking light detection and ranging (LiDAR), which achieves the multiple target detection, recognition, and data fusion. The You Only Look Once v4 (YOLOv4) network is utilized to achieve object detection and recognition on the color images. The depth images are obtained from the rectified left and right images based on the principle of the epipolar constraints, then the obstacles are detected from the depth images using the MeanShift algorithm. The pixel-level polarization images are extracted from the raw polarization-grey images, then the water hazards are detected successfully. The PointPillars network is employed to detect the objects from the point cloud. The calibration and synchronization between the sensors are accomplished. The experiment results show that the data fusion enriches the detection results, provides high-dimensional perceptual information and extends the effective detection range. Meanwhile, the detection results are stable under diverse range and illumination conditions.

Efficient three-dimensional point cloud object detection based on improved Complex-YOLO

“…Deep learning-related technologies are increasingly integrated into people’s daily life, and object detection algorithms ( Qi et al, 2021 ; Liu et al, 2022a , b ; Xu et al, 2022 ), as a crucial component of the autonomous driving perception layer, can create a solid foundation for behavioral judgments during autonomous driving. Although object detection algorithms based on 2D images ( Bochkovskiy et al, 2020 ; Bai et al, 2022 ; Cheon et al, 2022 ; Gromada et al, 2022 ; Long et al, 2022 ; Otgonbold et al, 2022 ; Wahab et al, 2022 ; Wang et al, 2022 ) have had a lot of success at this stage, single-view images cannot completely reflect the position pose, and motion orientation of objects in 3D space due to the lack of depth information in 2D images. Consequently, in the field of autonomous driving, the focus of object detection research has increasingly switched from 2D image detection to 3D image detection and point cloud detection.…”

Introductory Chapter: Autonomous Mobile Mapping Robots – Current State and Future Real-World Challenges

“…Due to an integrated IMU (Inertial Measurement Unit), it is possible to perform online lidar odometry [5] without any additional engineering effort. Thus, recent advances in mobile mapping algorithms show great improvement in lidar 3D mapping even looking from the perspective of other applications such ADAS (Advanced Driver Assistance Systems) [6] (related to future autonomous driving).…”