Learning to Remove Soft Shadows

Abstract

Manipulated images lose believability if the user's edits fail to account for shadows. We propose a method that makes removal and editing of soft shadows easy. Soft shadows are ubiquitous, but remain notoriously difficult to extract and manipulate. We posit that soft shadows can be segmented, and therefore edited, by learning a mapping function for image patches that generates shadow mattes. We validate this premise by removing soft shadows from photographs with only a small amount of user input.Given only broad user brush strokes that indicate the region to be processed, our new supervised regression algorithm automatically unshadows an image, removing the umbra and penumbra. The resulting lit image is frequently perceived as a believable shadow-free version of the scene. We tested the approach on a large set of soft shadow images, and performed a user study that compared our method to the state of the art and to real lit scenes. Our results are more difficult to identify as being altered, and are perceived as preferable compared to prior work.

DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal

“…These methods first locate the shadow regions either by shadow detection [15,19] or with user annotations [1,14,12,34]. Two reconstruction algorithms with hand-crafted features are then designed for removing the detected shadows from the umbra and penumbra regions.…”

“…Finally, they propose a Bayesian formulation to extract the shadow matte. Instead of using hand-crafted features to estimate the shadow matte, Gryka et al [14] propose a Random Forest based method to model the relationship between shadow image regions and their shadow matte. Although it is a data-driven method, it requires accurate shadow annotation and an initial guess of the shadow matte as input.…”

“…Existing methods for shadow matte estimation require the prior information of shadow location. It is either obtained from shadow detection [15,19] or user input [31,1,14,12,34]. However, shadow detection itself is a challenging task.…”

Appearance Harmonization for Single Image Shadow Removal

“…[1] estimate shadow density directly using patch lightness. Gryka et al [13] extract soft shadows by learning a regression function from image patches to shadow mattes. However, both these techniques use simple shadow models that are often inadequate for the non-linear images that are constitute the vast majority of photographs.…”

“…Improving previous results with texture inconsistency using SRH. (a) [7], (b) [23], (c) [11], (d) [10], (e) [13].…”