Self-Refining Deep Symmetry Enhanced Network for Rain Removal

Abstract

Rain removal aims to remove the rain streaks on rain images. The state-of-the-art methods are mostly based on Convolutional Neural Network (CNN). However, as CNN is not equivariant to object rotation, these methods are unsuitable for dealing with the tilted rain streaks. To tackle this problem, we propose Deep Symmetry Enhanced Network (DSEN) that is able to explicitly extract the rotation equivariant features from rain images. In addition, we design a self-refining mechanism to remove the accumulated rain streaks in a coarse-tofine manner. This mechanism reuses DSEN with a novel information link which passes the gradient flow to the higher stages. Extensive experiments on both synthetic and realworld rain images show that our self-refining DSEN yields the top performance.

A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

“…To extract long range, non-local contextual information, region-aware block [47], non-locally enhanced block [53] and spatial attentive block [46] are introduced. Some novel convolutional operations are also utilized, such as the dilated convolution for enlarging the receptive field [45,46,52], rotationally equivariant convolution [54], and paired operations (e.g. up and down-sampling) in dual residual connection [20].…”

Double Recurrent Dense Network for Single Image Deraining

“…Zhang et al [25] use the generative adversarial network (GAN) to prevent the degeneration of background when it is extracted from rain image, and utilized the perceptual loss to further ensure better visual quality. Liu et al [24] proposed a novel symmetry enhanced network to explicitly remove the tilted rain streaks from rain images.…”

“…Since there are no ground truth for real-world rainy images, the performance on the real-world dataset can only be evaluated in terms of visual effect. We compare our proposed approach with seven state-of-the-art methods, including image decomposition (ID) [1], discriminative sparse coding (DSC) [4], A directional global sparse model for single image rain removal [45], layer priors (LP) [5], DetailsNet [7], joint rain detection and removal (JORDER) [8], Single Image Rain Removal via a Simplified Residual Dense Network (SRDN) [38], Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal [23], Density-aware single image de-raining using a multi-stream dense network (DID-MDN) [12], Recurrent Squeeze-and-Excitation Context Aggregation Net(RESCAN) [10] and S-DSEN [24]. As for the task of image denoising, we also adopt PNSR and SSIM to compare the performances of our proposed model with six representative methods, including image denoising by sparse 3-d transform-domain collaborative filtering (BM3D) [33], Weighted nuclear norm minimization with application to image denoising (WNNM) [36], Image denoising: Can plain neural networks compete with BM3D (MLP) [37], A flexible framework for fast and effective image restoration (TNRD) [29], Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising (DnCNN) [35], Toward a fast and flexible solution for cnn based image denoising.…”

“…Since video-based methods can leverage the temporary information in adjacent frames, they are easier than the single-image based methods. More and more works by using the advantages of deep learning are proposed in recent years to remove the rain streaks from single image [8], [10], [24], [39]- [42].…”