3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions

Abstract

The role of sensors such as cameras or LiDAR (Light Detection and Ranging) is crucial for the environmental awareness of self-driving cars. However, the data collected from these sensors are subject to distortions in extreme weather conditions such as fog, rain, and snow. This issue could lead to many safety problems while operating a self-driving vehicle. The purpose of this study is to analyze the effects of fog on the detection of objects in driving scenes and then to propose methods for improvement. Collecting and processing data in adverse weather conditions is often more difficult than data in good weather conditions. Hence, a synthetic dataset that can simulate bad weather conditions is a good choice to validate a method, as it is simpler and more economical, before working with a real dataset. In this paper, we apply fog synthesis on the public KITTI dataset to generate the Multifog KITTI dataset for both images and point clouds. In terms of processing tasks, we test our previous 3D object detector based on LiDAR and camera, named the Spare LiDAR Stereo Fusion Network (SLS-Fusion), to see how it is affected by foggy weather conditions. We propose to train using both the original dataset and the augmented dataset to improve performance in foggy weather conditions while keeping good performance under normal conditions. We conducted experiments on the KITTI and the proposed Multifog KITTI datasets which show that, before any improvement, performance is reduced by 42.67% in 3D object detection for Moderate objects in foggy weather conditions. By using a specific strategy of training, the results significantly improved by 26.72% and keep performing quite well on the original dataset with a drop only of 8.23%. In summary, fog often causes the failure of 3D detection on driving scenes. By additional training with the augmented dataset, we significantly improve the performance of the proposed 3D object detection algorithm for self-driving cars in foggy weather conditions.

LossDistillNet: 3D Object Detection in Point Cloud Under Harsh Weather Conditions

“…Tabs. 8 shows comparison results between the LossDistill-Net and SLS-Fusion models [6]. SLS-Fusion [6] exhibits superior performance only in foggy weather conditions, as it uses a different fog simulator from that used in our study.…”

“…Although this model works well under normal weather conditions, its performance decreases significantly in foggy conditions. Accordingly, the model was modified in a subsequent study [6] by implementing a specific training strategy that uses both normal and foggy weather datasets to achieve higher performance. The study further points out that the late-fusion-based architecture can perform well with a justifiable training strategy in foggy weather conditions.…”

“…Our proposed architecture employs a probability estimation method to recover the lost data points. Specifically, by combining a DMFA network with a loss-convolution layer, our proposed model outperforms the models designed in [5] and [6].…”

Foggy Lane Dataset Synthesized from Monocular Images for Lane Detection Algorithms

“…However, the application scenarios of most deep learning (DL)-based lane detection are still limited to ideal weather conditions, e.g., clear daytime. However, little research [ 11 , 12 , 13 , 14 , 15 , 16 ] focuses on low-light weather conditions, such as foggy and rainy days, which are significant for increasing the adaptability of perception technology for autonomous driving vehicles. In fact, lane detection can be more challenging in complicated weather conditions.…”

“…The method of augmenting existing datasets by synthesized images has been used in several scenes, such as rainy [ 11 ] and nighttime [ 12 ] conditions. Despite some previous studies in synthesizing foggy images for autonomous driving, including FRIDA [ 13 ], FRIDA2 [ 14 ], Foggy CityScapes [ 15 ], and Multifog KITTI [ 16 ], there are few studies focusing on augmenting datasets using artificially generated foggy images and its effectiveness for lane detection task. Tarel et al constructed synthetic outdoor foggy datasets FRIDA (Foggy Road Image Database) and FRIDA2 based on numerical images using SiVIC TM software.…”