Improving the performance of automotive vision‐based applications under rainy conditions

Abstract

Input images are the main source of information for vision-based algorithms. The presence of raindrops in input images degrades their quality and, consequently, reduces the quality of the target vision-based algorithm that consumes them. Many image restoration algorithms were proposed in the literature to remove rain presence in images to improve the input image quality. These algorithms, however, cannot remove all the raindrop presence and sometimes introduce undesirable side-effects, such as the blurring rain-occluded sections of the image and incorrectly de-raining areas in the image that are clear. It is hypothesized that a comparable performance improvement can be achieved by decreasing the sensitivity of vision-based algorithms to noisy input images, rather than denoising these images, through the process of de-raining. To test this hypothesis, the performance of state-ofthe-art object detection and semantic segmentation models was evaluated, with de-rained image datasets used as input, and compared it to that performance of the same models, retrained with rained image sets. Results showed that the performance of the retrained models was better than that of the baseline detector with de-rained images used as input. INTRODUCTIONAutomotive systems including vision-based applications are highly regulated and are required to meet high performance and safety standards. This means that these systems must operate under all conditions, favourable or adverse. The quality of the system inputs has a direct impact on its performance, in the sense that noisy inputs result in degradation in system performance.Two approaches are usually implemented to reduce the effect of noisy inputs on system performance, denoising the inputs, or reducing system sensitivity to noise. Filtering analogue signals and debouncing digital ones are two examples of common input signal denoising techniques. Predictive modelling and sensor fusion are system design techniques that lead to reduced system sensitivity to noisy inputs.Rain is a type of adverse weather condition that degrades the quality of images and the performance of vision-based algorithms that consume them. In a previous research work [1], we showed that the performance of state-of-the-art object detectors (including YOLOv3, RCNN, and SSD) greatly degradesThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.