Interactive removal and ground truth for difficult shadow scenes

Abstract

A user-centric method for fast, interactive, robust, and high-quality shadow removal is presented. Our algorithm can perform detection and removal in a range of difficult cases, such as highly textured and colored shadows. To perform detection, an on-the-fly learning approach is adopted guided by two rough user inputs for the pixels of the shadow and the lit area. After detection, shadow removal is performed by registering the penumbra to a normalized frame, which allows us efficient estimation of nonuniform shadow illumination changes, resulting in accurate and robust removal. Another major contribution of this work is the first validated and multiscene category ground truth for shadow removal algorithms. This data set containing 186 images eliminates inconsistencies between shadow and shadow-free images and provides a range of different shadow types such as soft, textured, colored, and broken shadow. Using this data, the most thorough comparison of state-of-the-art shadow removal methods to date is performed, showing our proposed algorithm to outperform the state of the art across several measures and shadow categories. To complement our data set, an online shadow removal benchmark website is also presented to encourage future open comparisons in this challenging field of research.

Shadow Removal via Shadow Image Decomposition

“…Table 1: Shadow removal results of our networks compared to state-of-the-art shadow removal methods on the adjusted ground truth. ( * ) The method of Gong et al [12] is an interactive method that defines the shadow/nonshadow regions via user inputs, thus generates minimal error on the non-shadow area. The metric is RMSE (the lower, the better).…”

“…[38] [12] [34] (Ours) (Ours) Truth Figure 8: Comparison of shadow removal on ISTD dataset. Qualitative comparison between our method and previous state-of-the-art methods: Guo et al [13], Yang et al [38], Gong et al [12], and Wang et al [34]. "SP-Net" are the shadow removal results using the parameters computed from SP-Net and a binary shadow mask.…”

“…The shadow matte represents the per-pixel linear combination of the relit image and the shadow image, which results to the shadow-free image. Previous work often requires user assistance [12] or solving an optimization system [20] to obtain the shadow mattes. In contrast, we propose to train a second network (M-Net) to accurately predict shadow mattes in a fully automated manner.…”

“…We train and test our proposed SP-Net and M-Net on the ISTD dataset [34], which is the largest and most challenging available dataset for shadow removal. SP-Net alone (no matting) outperforms the state-of-the-art [12] in shadow removal by 29% in terms of RMSE on shadow areas, from 13.3 to 9.5 RMSE. Our full system with both SP-Net and M-Net further improves the overall results by another 17%, which yields a RMSE of 7.9.…”

Image Shadow Removal Using End-To-End Deep Convolutional Neural Networks

“…Existing methods of removing shadow regions typically involve two steps: shadow detection and shadow removal. Firstly, shadow detection is used to locate the shadow area [13,14] or the user manually marks the shadow area [9,15,16], then the model is constructed to rebuild both and to remove shadows.…”