Direction-Aware Spatial Context Features for Shadow Detection and Removal

Abstract

Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features in different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a Euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistency in the training pairs. We employ two shadow detection benchmark datasets and two shadow removal benchmark datasets, and perform various experiments to evaluate our method. Experimental results show that our method clearly outperforms state-of-the-art methods for both shadow detection and shadow removal.

ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal

“…we visualize some results in Figure 7. As we can see, (1) traditional methods Guo [12] and Zhang [60] are not able to effectively detect slender shadows in the image; (2) among all deep learning methods, comparing with ST-CGAN [54], DSC [16], A+D Net [27] and BDRAR [63], our proposed ARGAN is able to detect more accurate shadow regions and even more close to our human observation. Figure 8 presents two more shadow images with more complex scenes.…”

“…One is Qu et al's multi-context embedding network [45] integrating high-level semantic context for shadow removal. One is Hu et al's [16] using direction-aware spatial context features for shadow detection and removal. Another is Wang et al's GAN-based method [54] which jointly learns shadow detection and shadow removal.…”

Mask-ShadowGAN: Learning to Remove Shadows From Unpaired Data

“…Evaluation metrics. We followed recent works [13,27,35] to evaluate shadow removal performance by computing the root-mean-square error (RMSE) between the ground truth and predicted shadow-free images in LAB color space. In general, a small RMSE indicates a better performance.…”

“…First, we trained our model on the USR training set and applied it to produce shadow-free images on the USR testing set. Also, we applied several state-ofthe-art methods to remove shadows on the USR testing set: DSC [13], Gong et al [8], and Guo et al [11]. For DSC, we adopted its public implementation and trained its network on the SRD and ISTD datasets: "DSC-S" and "DSC- [13,35] 4.78 ± 2.92 DSC-S [13,27] 4.60 ± 2.66 Gong et al [8] 2.82 ± 1.76 Guo et al [11] 2.31 ± 1.90…”

“…Mask-ShadowGAN DSC [13] ST-CGAN [35] Gong et al [8] Guo et al [11] input ground truth Mask-ShadowGAN DSC [13] DeshadowNet [27] Gong et al [8] Guo et al [11] Figure 5: Comparing shadow removal results on the ISTD [35] (top two rows) & SRD [27] (last three rows) datasets. In the fifth column, the top two results are produced from ST-CGAN [35] while the others are produced from DeshadowNet [27].…”