L-DIG: A GAN-Based Method for LiDAR Point Cloud Processing under Snow Driving Conditions

Abstract

LiDAR point clouds are significantly impacted by snow in driving scenarios, introducing scattered noise points and phantom objects, thereby compromising the perception capabilities of autonomous driving systems. Current effective methods for removing snow from point clouds largely rely on outlier filters, which mechanically eliminate isolated points. This research proposes a novel translation model for LiDAR point clouds, the ‘L-DIG’ (LiDAR depth images GAN), built upon refined generative adversarial networks (GANs). This model not only has the capacity to reduce snow noise from point clouds, but it also can artificially synthesize snow points onto clear data. The model is trained using depth image representations of point clouds derived from unpaired datasets, complemented by customized loss functions for depth images to ensure scale and structure consistencies. To amplify the efficacy of snow capture, particularly in the region surrounding the ego vehicle, we have developed a pixel-attention discriminator that operates without downsampling convolutional layers. Concurrently, the other discriminator equipped with two-step downsampling convolutional layers has been engineered to effectively handle snow clusters. This dual-discriminator approach ensures robust and comprehensive performance in tackling diverse snow conditions. The proposed model displays a superior ability to capture snow and object features within LiDAR point clouds. A 3D clustering algorithm is employed to adaptively evaluate different levels of snow conditions, including scattered snowfall and snow swirls. Experimental findings demonstrate an evident de-snowing effect, and the ability to synthesize snow effects.

LiDAR Point Cloud Augmentation for Adverse Conditions Using Conditional Generative Model

“…We manually identified and labeled all snow clusters that are discernible to the human eye. A 3D clustering algorithm based on OPTICS (ordering points to identify the clustering structure) [6,27] is then employed to aggregate and analyze their spatial and clustering characteristics. An adaptive cluster amount determination step based on DBSCAN [28] was added before the OPTICS algorithm to improve the clustering efficiency.…”

“…With this mechanism, the possibility of one object being divided into several clusters or several objects being partitioned into one cluster is greatly minimized, leading to the result of the most appropriate number of cluster distributions. The 3D clustering algorithm produces seven metrics that could help the quantitative evaluation, as employed in our previous work [6]. Among these, reachability distance and cluster size are particularly useful for producing cluster-based segmentation maps as shown in Figure 1.…”

“…This classification equates to a segmentation map with four distinct labels, effectively illuminating the presence of various adverse conditions. We adopt the same RGB depth image format in our previous work [6,34], which represents the perpendicular projection of point clouds while keeping the depth information (i.e., the distances from points to the ego vehicle) within its pixel value, while the RGB channels now store labels in the segmentation map datasets. In this map, Labels 1 and 2 are assigned to noise clusters and individual noise points, stored in the red and blue channels, respectively.…”

“…Incorporating a 4-ResNet-block structure based on the GAN structure [6,13], the input channels of the generator G are adjusted to accommodate the six-channel input. This modification facilitates the integration of the concatenated data.…”

“…This is largely due to the inherent difficulty and unreliability of annotating or labeling adverse conditions [5] and the lack of samples featuring adverse conditions within current common driving datasets [1]. Furthermore, datasets specifically targeted at collecting adverse conditions in certain areas, like snowy regions, often face significant domain gap issues when being input into perception models trained with general data [6]. As a result, augmenting clear condition datasets with simulated adverse conditions has become one of the most effective and popular approaches to enhance the capability of deep-learning-based models in addressing adverse challenges.…”