Single Image De-Raining via Improved Generative Adversarial Nets

Outdoor vision sensing systems often struggle with poor weather conditions, such as snow and rain, which poses a great challenge to existing video desnowing and deraining methods. In this paper, we propose a novel video desnowing and deraining model that utilizes the salience information of moving objects to address this problem. First, we remove the snow and rain from the video by low-rank tensor decomposition, which makes full use of the spatial location information and the correlation between the three channels of the color video. Second, because existing algorithms often regard sparse snowflakes and rain streaks as moving objects, this paper injects salience information into moving object detection, which reduces the false alarms and missed alarms of moving objects. At the same time, feature point matching is used to mine the redundant information of moving objects in continuous frames, and a dual adaptive minimum filtering algorithm in the spatiotemporal domain is proposed by us to remove snow and rain in front of moving objects. Both qualitative and quantitative experimental results show that the proposed algorithm is more competitive than other state-of-the-art snow and rain removal methods.

Section: Single Image Snow and Rain Removal Methodsmentioning

confidence: 99%

Video Desnowing and Deraining via Saliency and Dual Adaptive Spatiotemporal Filtering

Jia

et al. 2021

“…Influenced by the different shapes and refractive indexes, the image content seen through raindrops depends on the raindrop occluded background and the whole environment [ 21 ]. The raindrops with high transparency refractive indexes tend to produce remarkably different semantics from the occluded background, which have serious impacts on raindrop removal and are neglected in [ 7 , 10 ]. To address this problem, we introduce a multi-scale dilated convolution module (MDCM) in the middle junction of the encoder and decoder, as shown in Figure 3 .…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Previous studies on rain removal have achieved great progress and have mainly focused on rain streaks [ 5 , 6 , 7 , 8 , 9 , 10 ] and rain mist [ 11 , 12 ]. Since the image formation and physical properties of raindrops are very different from those of rain streaks and rain mist, previous methods cannot be applied directly to raindrop removal.…”

Section: Introductionmentioning

confidence: 99%

Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image

Luo

et al. 2020

Removing raindrops from a single image is a challenging problem due to the complex changes in shape, scale, and transparency among raindrops. Previous explorations have mainly been limited in two ways. First, publicly available raindrop image datasets have limited capacity in terms of modeling raindrop characteristics (e.g., raindrop collision and fusion) in real-world scenes. Second, recent deraining methods tend to apply shape-invariant filters to cope with diverse rainy images and fail to remove raindrops that are especially varied in shape and scale. In this paper, we address these raindrop removal problems from two perspectives. First, we establish a large-scale dataset named RaindropCityscapes, which includes 11,583 pairs of raindrop and raindrop-free images, covering a wide variety of raindrops and background scenarios. Second, a two-branch Multi-scale Shape Adaptive Network (MSANet) is proposed to detect and remove diverse raindrops, effectively filtering the occluded raindrop regions and keeping the clean background well-preserved. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art raindrop removal methods. Moreover, the extension of our method towards the rainy image segmentation and detection tasks validates the practicality of the proposed method in outdoor applications.

“…Single-image rain removal methods [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ] can be categorized into model-based and data-driven methods. Model-based approaches employ optimization frameworks consisting of a data fidelity term and a prior term.…”

Section: Related Workmentioning

confidence: 99%

“…Since the introduction of the detailed network [ 13 ], various architectures such as density-aware [ 14 ], joint rain detection and removal [ 7 , 15 ], scale-aware [ 16 ], progressive networks [ 17 ], and attention models [ 18 , 19 ] have been developed. However, these deep learning models target rain streaks and heavy rain images.…”

Section: Related Workmentioning

confidence: 99%

Heavy Rain Face Image Restoration: Integrating Physical Degradation Model and Facial Component-Guided Adversarial Learning

Son¹,

Jeong²

2022

With the recent increase in intelligent CCTVs for visual surveillance, a new image degradation that integrates resolution conversion and synthetic rain models is required. For example, in heavy rain, face images captured by CCTV from a distance have significant deterioration in both visibility and resolution. Unlike traditional image degradation models (IDM), such as rain removal and super resolution, this study addresses a new IDM referred to as a scale-aware heavy rain model and proposes a method for restoring high-resolution face images (HR-FIs) from low-resolution heavy rain face images (LRHR-FI). To this end, a two-stage network is presented. The first stage generates low-resolution face images (LR-FIs), from which heavy rain has been removed from the LRHR-FIs to improve visibility. To realize this, an interpretable IDM-based network is constructed to predict physical parameters, such as rain streaks, transmission maps, and atmospheric light. In addition, the image reconstruction loss is evaluated to enhance the estimates of the physical parameters. For the second stage, which aims to reconstruct the HR-FIs from the LR-FIs outputted in the first stage, facial component-guided adversarial learning (FCGAL) is applied to boost facial structure expressions. To focus on informative facial features and reinforce the authenticity of facial components, such as the eyes and nose, a face parsing-guided generator and facial local discriminators are designed for FCGAL. The experimental results verify that the proposed approach based on a physical-based network design and FCGAL can remove heavy rain and increase the resolution and visibility simultaneously. Moreover, the proposed heavy rain face image restoration outperforms state-of-the-art models of heavy rain removal, image-to-image translation, and super resolution.