A Review of Detection and Removal of Raindrops in Automotive Vision Systems

Hamzeh, Yazan; Rawashdeh, Samir A.

doi:10.3390/jimaging7030052

Cited by 13 publications

(7 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the research work (see, for example, [2][3][4]) is focused on image restoration of rained images, by applying a de-raining process on them. As we have shown in our survey paper on adherent raindrop removal techniques [5], none of the reviewed de-raining algorithms could perfectly restore the rained images to resemble the clear ones. The improvements in deep-learning and convolution neural networks (CNN) opened the door for a new set of de-raining techniques that, generally, achieved better performance levels compared with classical machine learning algorithms.…”

mentioning

confidence: 84%

Improving the performance of automotive vision‐based applications under rainy conditions

Hamzeh

Mohammadi

Rawashdeh

2022

IET Image Processing

Self Cite

View full text Add to dashboard Cite

Input images are the main source of information for vision-based algorithms. The presence of raindrops in input images degrades their quality and, consequently, reduces the quality of the target vision-based algorithm that consumes them. Many image restoration algorithms were proposed in the literature to remove rain presence in images to improve the input image quality. These algorithms, however, cannot remove all the raindrop presence and sometimes introduce undesirable side-effects, such as the blurring rain-occluded sections of the image and incorrectly de-raining areas in the image that are clear. It is hypothesized that a comparable performance improvement can be achieved by decreasing the sensitivity of vision-based algorithms to noisy input images, rather than denoising these images, through the process of de-raining. To test this hypothesis, the performance of state-ofthe-art object detection and semantic segmentation models was evaluated, with de-rained image datasets used as input, and compared it to that performance of the same models, retrained with rained image sets. Results showed that the performance of the retrained models was better than that of the baseline detector with de-rained images used as input. INTRODUCTIONAutomotive systems including vision-based applications are highly regulated and are required to meet high performance and safety standards. This means that these systems must operate under all conditions, favourable or adverse. The quality of the system inputs has a direct impact on its performance, in the sense that noisy inputs result in degradation in system performance.Two approaches are usually implemented to reduce the effect of noisy inputs on system performance, denoising the inputs, or reducing system sensitivity to noise. Filtering analogue signals and debouncing digital ones are two examples of common input signal denoising techniques. Predictive modelling and sensor fusion are system design techniques that lead to reduced system sensitivity to noisy inputs.Rain is a type of adverse weather condition that degrades the quality of images and the performance of vision-based algorithms that consume them. In a previous research work [1], we showed that the performance of state-of-the-art object detectors (including YOLOv3, RCNN, and SSD) greatly degradesThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.

show abstract

mentioning

confidence: 84%

Improving the performance of automotive vision‐based applications under rainy conditions

Hamzeh

Mohammadi

Rawashdeh

2022

IET Image Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Before new LiDAR technology emerges besides 1550 nm wavelength, camera is still the major focus of perception enhancement in rain, mainly in terms of de-raining technique, which has been deeply studied by the computer vision field. The detection and removal of raindrops can be divided into falling raindrops and adherent raindrops that accumulated on the protective covers of cameras [164]. For rain streaks removal, several training and learning methods have been put to use including Quasi-Sparsity-based training [159] and continual learning [160].…”

Section: Rainmentioning

confidence: 99%

“…Histogram of oriented gradient (HOG) and autocorrelation loss are used to facilitate the orientation consistency and repress repetitive rain streaks. They trained the network all the way from drizzle to downpour rain Fusion [110] LiDAR [152] LiDAR [76] LiDAR [153] Others [154] LiDAR [155] LiDAR [156] Camera [157] Camera [158] Camera [159] Camera [160] Camera [161] Camera [162] Camera [163] Camera [164] LiDAR [165] LiDAR [166] LiDAR [128] LiDAR [29] Fusion [129] LiDAR [167] Fusion [168] LiDAR [169] LiDAR [170] Fusion [171] LiDAR [172] Camera [173] Camera [174] Camera [175] Camera [176] Camera [177] Camera [178] Camera [179] Camera [180] Camera [181] Camera [182] Camera [183] Camera [184] Camera [185] Camera [186] Fusion [187] Fusion [188] LiDAR [189] Camera [190] Camera…”

Section: Rainmentioning

confidence: 99%

“…Detection of adherent raindrops seems to be easy to achieve given the presumed optical conditions are met, but real-time removal of adherent raindrops inevitably brings the trade-off of processing latencies regardless of the performance. Peak signal-tonoise ratio (PSNR) and Structural Similarity (SSIM) metrics although widely implemented and faintly improved, don't seem exactly promising in the line of raindrops detection and removal compared to deep-learning and convolutional neural networks (CNN) [164].…”

Section: Rainmentioning

confidence: 99%

See 1 more Smart Citation

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Zhang¹,

Carballo²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automated Driving Systems (ADS) open up a new domain for the automotive industry and offer new possibilities for future transportation with higher efficiency and comfortable experiences. However, autonomous driving under adverse weather conditions has been the problem that keeps autonomous vehicles (AVs) from going to level 4 or higher autonomy for a long time. This paper assesses the influences and challenges that weather brings to ADS sensors in an analytic and statistical way, and surveys the solutions against inclement weather conditions. Stateof-the-art techniques on perception enhancement with regard to each kind of weather are thoroughly reported. External auxiliary solutions like V2X technology, weather conditions coverage in currently available datasets, simulators, and experimental facilities with weather chambers are distinctly sorted out. By pointing out all kinds of major weather problems the autonomous driving field is currently facing, and reviewing both hardware and computer science solutions in recent years, this survey contributes a holistic overview on the obstacles and directions of ADS development in terms of adverse weather driving conditions.

show abstract

“…Consequently, more precise and quicker target detection technologies are required to lower the danger of pedestrian and vehicle collisions under inclement weather conditions. In target detection tasks, inclement weather conditions can significantly impact the performance of image and video-based traffic analysis systems (Hamzeh and Rawashdeh, 2021 ). Histograms of Oriented Gradients (Arróspide et al, 2013 ), the Deformable Part Model (Cai et al, 2017 ), the Viola-Jones (Xu et al, 2016 ), and so forth.…”

Section: Introductionmentioning

confidence: 99%

Real-time vehicle target detection in inclement weather conditions based on YOLOv4

Wang

Zhao

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

As a crucial component of the autonomous driving task, the vehicle target detection algorithm directly impacts driving safety, particularly in inclement weather situations, where the detection precision and speed are significantly decreased. This paper investigated the You Only Look Once (YOLO) algorithm and proposed an enhanced YOLOv4 for real-time target detection in inclement weather conditions. The algorithm uses the Anchor-free approach to tackle the problem of YOLO preset anchor frame and poor fit. It better adapts to the detected target size, making it suitable for multi-scale target identification. The improved FPN network transmits feature maps to unanchored frames to expand the model's sensory field and maximize the utilization of model feature data. Decoupled head detecting head to increase the precision of target category and location prediction. The experimental dataset BDD-IW was created by extracting specific labeled photos from the BDD100K dataset and fogging some of them to test the proposed method's practical implications in terms of detection precision and speed in Inclement weather conditions. The proposed method is compared to advanced target detection algorithms in this dataset. Experimental results indicated that the proposed method achieved a mean average precision of 60.3%, which is 5.8 percentage points higher than the original YOLOv4; the inference speed of the algorithm is enhanced by 4.5 fps compared to the original, reaching a real-time detection speed of 69.44 fps. The robustness test results indicated that the proposed model has considerably improved the capacity to recognize targets in inclement weather conditions and has achieved high precision in real-time detection.

show abstract

A Review of Detection and Removal of Raindrops in Automotive Vision Systems

Cited by 13 publications

References 63 publications

Improving the performance of automotive vision‐based applications under rainy conditions

Improving the performance of automotive vision‐based applications under rainy conditions

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Real-time vehicle target detection in inclement weather conditions based on YOLOv4

Contact Info

Product

Resources

About