Joint Convolutional Analysis and Synthesis Sparse Representation for Single Image Layer Separation

Gu, Shuhang; Meng, Deyu; Zuo, Wangmeng; Zhang, Lei

doi:10.1109/iccv.2017.189

Cited by 202 publications

(171 citation statements)

References 31 publications

Supporting

Mentioning

171

Contrasting

Order By: Relevance

“…Similarly, Zhang et al [30] learned a set of generic sparsitybased and low-rank representation-based convolutional filters to represent background and rain streaks, respectively. Gu et al [14] combined analysis sparse representation to represent image large-scale structures and synthesis sparse representation to represent image fine-scale textures, including the directional prior and the non-negativeness prior in their JCAS model. More recently, Zhu et al [33] proposed a joint optimization process that alternates between removing rain-streak details from background layer and removing non-streak details from rain layer.…”

Section: Related Work 21 Single Image Rain Removal Methodsmentioning

confidence: 99%

Semi-Supervised Transfer Learning for Image Rain Removal

Wei¹,

Meng

Zhao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

366

317

View full text Add to dashboard Cite

Single image rain removal is a typical inverse problem in computer vision. The deep learning technique has been verified to be effective for this task and achieved state-of-theart performance. However, previous deep learning methods need to pre-collect a large set of image pairs with/without synthesized rain for training, which tends to make the neural network be biased toward learning the specific patterns of the synthesized rain, while be less able to generalize to real test samples whose rain types differ from those in the training data. To this issue, this paper firstly proposes a semi-supervised learning paradigm toward this task. Different from traditional deep learning methods which only use supervised image pairs with/without synthesized rain, we further put real rainy images, without need of their clean ones, into the network training process. This is realized by elaborately formulating the residual between an input rainy image and its expected network output (clear image without rain) as a specific parametrized rain streaks distribution. The network is therefore trained to adapt real unsupervised diverse rain types through transferring from the supervised synthesized rain, and thus both the short-of-training-sample and bias-to-supervised-sample issues can be evidently alleviated. Experiments on synthetic and real data verify the superiority of our model compared to the state-of-the-arts.

show abstract

Section: Related Work 21 Single Image Rain Removal Methodsmentioning

confidence: 99%

Semi-Supervised Transfer Learning for Image Rain Removal

Wei¹,

Meng

Zhao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

366

317

View full text Add to dashboard Cite

show abstract

“…Luo et al [5] propose a discriminative sparse coding framework based on image patches. Gu et al [23] integrate analysis sparse representation (ASR) and synthesis sparse representation (SSR) to solve a variety of image decomposition problems. In [7], GMM works as a prior to decompose a rain image into background and rain streaks layer.…”

Section: Related Workmentioning

confidence: 99%

Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

Lin

et al. 2018

Lecture Notes in Computer Science

649

599

View full text Add to dashboard Cite

Rain streaks can severely degrade the visibility, which causes many current computer vision algorithms fail to work. So it is necessary to remove the rain from images. We propose a novel deep network architecture based on deep convolutional and recurrent neural networks for single image deraining. As contextual information is very important for rain removal, we first adopt the dilated convolutional neural network to acquire large receptive field. To better fit the rain removal task, we also modify the network. In heavy rain, rain streaks have various directions and shapes, which can be regarded as the accumulation of multiple rain streak layers. We assign different alpha-values to various rain streak layers according to the intensity and transparency by incorporating the squeeze-and-excitation block. Since rain streak layers overlap with each other, it is not easy to remove the rain in one stage. So we further decompose the rain removal into multiple stages. Recurrent neural network is incorporated to preserve the useful information in previous stages and benefit the rain removal in later stages. We conduct extensive experiments on both synthetic and real-world datasets. Our proposed method outperforms the state-of-the-art approaches under all evaluation metrics. Codes and supplementary material are available at our project webpage: https://xialipku.github.io/RESCAN.

show abstract

“…Here, we choose the better performed result between dehaze+derain and derain+dehaze for those rain streaks removal methods. [10][7][21] [40]. Note that directly using GAN method such as [13] [42] does not produce appropriate solution for this image enhancement problem since these generative models can sometimes generate fake results as shown in the first example (top part) of Fig.5.…”

Section: Synthetic Rain Analysismentioning

confidence: 99%

Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning

Cheong

Tan

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

292

305

View full text Add to dashboard Cite

Most deraining works focus on rain streaks removal but they cannot deal adequately with heavy rain images. In heavy rain, streaks are strongly visible, dense rain accumulation or rain veiling effect significantly washes out the image, further scenes are relatively more blurry, etc. In this paper, we propose a novel method to address these problems. We put forth a 2-stage network: a physics-based backbone followed by a depth-guided GAN refinement. The first stage estimates the rain streaks, the transmission, and the atmospheric light governed by the underlying physics. To tease out these components more reliably, a guided filtering framework is used to decompose the image into its lowand high-frequency components. This filtering is guided by a rain-free residue image -its content is used to set the passbands for the two channels in a spatially-variant manner so that the background details do not get mixed up with the rain-streaks. For the second stage, the refinement stage, we put forth a depth-guided GAN to recover the background details failed to be retrieved by the first stage, as well as correcting artefacts introduced by that stage. We have evaluated our method against the state of the art methods. Extensive experiments show that our method outperforms them on real rain image data, recovering visually clean images with good details. A T J = (I (1T)*A)/TS S Depth Guidance Physics-Based StageModel-Free Stage High-PassLow-Pass

show abstract

Joint Convolutional Analysis and Synthesis Sparse Representation for Single Image Layer Separation

Cited by 202 publications

References 31 publications

Semi-Supervised Transfer Learning for Image Rain Removal

Semi-Supervised Transfer Learning for Image Rain Removal

Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning

Contact Info

Product

Resources

About