Video Desnowing and Deraining via Saliency and Dual Adaptive Spatiotemporal Filtering

Abstract

Outdoor vision sensing systems often struggle with poor weather conditions, such as snow and rain, which poses a great challenge to existing video desnowing and deraining methods. In this paper, we propose a novel video desnowing and deraining model that utilizes the salience information of moving objects to address this problem. First, we remove the snow and rain from the video by low-rank tensor decomposition, which makes full use of the spatial location information and the correlation between the three channels of the color video. Second, because existing algorithms often regard sparse snowflakes and rain streaks as moving objects, this paper injects salience information into moving object detection, which reduces the false alarms and missed alarms of moving objects. At the same time, feature point matching is used to mine the redundant information of moving objects in continuous frames, and a dual adaptive minimum filtering algorithm in the spatiotemporal domain is proposed by us to remove snow and rain in front of moving objects. Both qualitative and quantitative experimental results show that the proposed algorithm is more competitive than other state-of-the-art snow and rain removal methods.

A review of video‐based rainfall measurement methods

“…The raindrop‐recognition process can be formulated as There are three categories of raindrop recognition algorithms (Yang et al, 2021): time domain‐based, low rank and sparsity‐based, and deep learning‐based methods. The first two categories leverage the prior knowledge of the rain and the background and thus represent model‐based methodologies, where is a spatial–temporal filter (Li et al, 2021) or sparse representation (Kim et al, 2015), and so forth. The final category is a data‐driven methodology, where represents the designed neural network, which exploits hierarchical features to obtain more complicated mappings between rain and clean images (Yang et al, 2021).…”

Decomposed Dissimilarity Measure for Evaluation of Digital Image Denoising

“…One of the typical problems of such characteristics is image denoising, especially impulsive noise suppression, which has been recently explored and become a popular topic to examine [ 7 , 8 , 9 , 10 , 11 , 12 , 13 ]. Other applications, where larger artifacts are detected and removed from the images but not all the pixels are altered, include rain [ 14 , 15 , 16 , 17 , 18 , 19 , 20 ], snow [ 20 , 21 , 22 , 23 , 24 ], marine snow [ 25 ], and crack [ 26 ] removal problems. Furthermore, algorithms that are dedicated to image tamper detection and correction [ 27 ] may fit into the above-mentioned description.…”