Mask-ShadowGAN: Learning to Remove Shadows From Unpaired Data

Abstract

This paper presents a new method for shadow removal using unpaired data, enabling us to avoid tedious annotations and obtain more diverse training samples. However, directly employing adversarial learning and cycleconsistency constraints is insufficient to learn the underlying relationship between the shadow and shadow-free domains, since the mapping between shadow and shadow-free images is not simply one-to-one. To address the problem, we formulate Mask-ShadowGAN, a new deep framework that automatically learns to produce a shadow mask from the input shadow image and then takes the mask to guide the shadow generation via re-formulated cycle-consistency constraints. Particularly, the framework simultaneously learns to produce shadow masks and learns to remove shadows, to maximize the overall performance. Also, we prepared an unpaired dataset for shadow removal and demonstrated the effectiveness of Mask-ShadowGAN on various experiments, even it was trained on unpaired data.

Pyramid Spatial Context Features for Salient Object Detection

“…Especially, our method shows a large improvement on the DUTS-test dataset, which includes many challenge cases, demonstrating the strong capability of our PSCNet. In the future, we will explore the potential of our PSC module design for other layer separation tasks, such as mirror detection [79], lane marking detection [80], shadow detection [81]- [83] and removal [84], [85], reflection removal [86], [87], rain removal [88], haze removal [89], [90], etc.…”

Image Shadow Removal Based on Generative Adversarial Networks

“…As we outlined above generator network G f→ s additionally takes attention map A s and generated mask M s [9] as the input (concatenating to the image as additional channels). To preserve the consistency between the generated shadow image and the original one we take the same attention map and shadow mask extracted from shadow removal generator G s→ f .…”

“…Our generator G f→ s uses the shadow mask as the input, so we can condition network with it and generate multiple shadows from one shadow-free image. We follow the same approach as [9] and construct the threshold binarizer B between generated shadow free image If and original image I s :…”

“…One of the early works [21] used three different neural networks operating in different contexts for more quality features extraction. Later, Hu et al [9] explored direction-aware spatial context for this task to compensate the lack of data. After that, Wang et al [24] used adversarial learning by stacking two networks together where one is used for shadow detection and one for shadow removal.…”

“…Recently, the unsupervised solution was presented [27] where the problem was formulated in unsupervised mapping learning between two domains -shadow and shadow-free -using CycleGAN [9] framework. First CGAN detects and removes the shadow while the other tries to generate it given the image and shadow mask as the input.…”