Single Image Reflection Removal Based on GAN With Gradient Constraint

Abstract

When we take a picture through glass windows, the photographs are often degraded by undesired reflections. To separate reflection layer and background layer is an important problem for enhancing image quality. However, single-image reflection removal is a challenging process because of the ill-posed nature of the problem. In this paper, we propose a single-image reflection removal method based on generative adversarial networks. Our network is an end-to-end trained network with four types of losses. It includes pixel loss, feature loss, adversarial loss, and gradient constraint loss. We propose a novel gradient constraint loss in order to separate the background layer and the reflection layer clearly. Gradient constraint loss is applied in a gradient domain and it minimizes the correlation between the background and reflection layer. Owing to the novel loss and our new synthetic dataset, our reflection removal method outperforms state-of-the-art methods in PSNR and SSIM, especially in real world images.

Single-Image Reflection Removal Using Deep Learning: A Systematic Review

“…ERR Net [9] was proposed by Wei et al which is able to be trained by misaligned data. [1]. In addition to these papers that we mentioned in the current paragraph, many other research papers are explored in this systematic review to find comprehensive answers for our proposed questions.…”

“…This method has been utilized in a great number of previous mathematical techniques for quantitative evaluations [5], [61], [62], [63], [64]. In the process of synthesizing the dataset, they randomly flipped the images and another image is utilized as a background layer and another image is utilized as the reflection layer [1]. After reviewing all of the considered articles, we summarized the datasets which were used with the purpose of training the deep learning algorithm for each research paper in table 7.…”

“… In P12, Abiko et al [1] proposed an algorithm based on GAN with gradient constraint. This algorithm has three sub-networks Generator, discriminator, and feature extractor that are operating based on CNN.…”

“…When we capture images through transparent material especially glass, the images frequently consist of some undesirable reflections. These reflections will lower the quality and visibility of the images [1], [2]. Some photographers may make a dark condition or modify the location of the camera but this approach is not efficient for reflection removal due to the limitations on space.…”

“…Some photographers may make a dark condition or modify the location of the camera but this approach is not efficient for reflection removal due to the limitations on space. These reflections have two main problems: first, they will lower the quality of the scenes behind the glass and second, they have some effects on the results of applications like segmentation or classification [1]. Therefore, this problem may cause most computer vision algorithms presumably to fail.…”