Style-Guided Shadow Removal

Jin, Wei; Yin, Hui; Wu, Zhenyao; Wu, Xinyi; Liu, Yanting; Wang, Song

doi:10.1007/978-3-031-19800-7_21

Cited by 17 publications

(15 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baseline methods for comparison. To further demonstrate the advantages of the structure-informed shadow removal networks, we compare the StructNet variants and MStructNet with state-of-the-art methods including Guo et al [6], Gong et al [7], DeshadwoNet [9], STCGAN [11], DSC [10], Mask-ShadowGAN [13], AR-GAN [53], SP+M-Net [33], CLA-GAN [25], RIS-GAN [26], Param+M+D-Net [12], DHAN [24], G2R [29], AEF [8], DC-ShadowGAN [30], SP+M+I-Net [34], BMNet [27], SADC [17], EMD-Net [36], and SG-ShadowNet [28]. Since the settings of different methods (e.g., test environment and inference resolution) TABLE 2: Validation results of StructNet-equipped shadow removal methods on ISTD and ISTD+ datasets.…”

Section: Baseline Methodsmentioning

confidence: 99%

“…It uses a multi-branch CNNs to extract multi-level contexts for shadow removal. Since then, many methods [10], [11], [24], [25], [26], [27], [28] have been proposed following this image-to-image mapping paradigm. They focus on designing intricate network architectures and exploiting distinctive properties (e.g., contexts, residuals and illuminations).…”

Section: Shadow Removalmentioning

confidence: 99%

“…To avoid the influence of the fusion function carried out by the MFRA module, we replace it with a naive element-wise additive operation instead. We 13.3 --STCGAN [11] 9.4 4.3 5.1 SP+M-Net [33] 7.9 3.1 3.9 Param+M+D-Net [12] 9.7 3.0 4.0 G2R (w sup) [29] 7.3 2.9 3.6 AEF [8] 6.5 3.8 4.2 DC-ShadowGAN [30] 10.3 3.5 4.6 SP+M+I-Net [34] 6.0 3.1 3.6 BMNet [27] 6.1 2.9 3.5 SADC [17] 6.2 3.1 3.6 SG-ShadowNet [28] 5.9 2.9 3.4 shadow regions, which demonstrates that the MSFE does benefit the structure-level shadow removal and enhance the image-level shadow removal. In general, if we embed MSFE in a deeper convolution layer, we get lower RMSEs in the shadow regions while higher RMSEs in the nonshadow regions at the structure level.…”

Section: Effectiveness Of the Msfe Modulementioning

confidence: 99%

See 2 more Smart Citations

Structure-Informed Shadow Removal Networks

Wang¹,

Guo²,

Fu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Shadow removal is a fundamental task in computer vision. Despite the success, existing deep learning-based shadow removal methods still produce images with shadow remnants. These shadow remnants typically exist in homogeneous regions with low intensity values, making them untraceable in the existing image-to-image mapping paradigm. We observe from our experiments that shadows mainly degrade object colors at the image structure level (in which humans perceive object outlines filled with continuous colors). Hence, in this paper, we propose to remove shadows at the image structure level. Based on this idea, we propose a novel structure-informed shadow removal network (StructNet) to leverage the image structure information to address the shadow remnant problem. Specifically, StructNet first reconstructs the structure information of the input image without shadows, and then uses the restored shadow-free structure prior to guiding the image-level shadow removal. StructNet contains two main novel modules: (1) a mask-guided shadow-free extraction (MSFE) module to extract image structural features in a non-shadow to shadow directional manner, and (2) a multi-scale feature & residual aggregation (MFRA) module to leverage the shadow-free structure information to regularize feature consistency. In addition, we also propose to extend StructNet to exploit multi-level structure information (MStructNet), to further boost the shadow removal performance with minimum computational overheads. Extensive experiments on three shadow removal benchmarks demonstrate that our method outperforms existing shadow removal methods, and our StructNet can be integrated with existing methods to further boost their performances.

show abstract

Section: Baseline Methodsmentioning

confidence: 99%

Section: Shadow Removalmentioning

confidence: 99%

Section: Effectiveness Of the Msfe Modulementioning

confidence: 99%

See 1 more Smart Citation

Structure-Informed Shadow Removal Networks

Wang¹,

Guo²,

Fu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…SpA‐Former [ZGZ22] shows insensitivity to shadow regions and fails to remove complex facial shadows effectively, as shown in Figure 9(e). SG‐ShadowNet [WYW*22] can remove shadows in the face but introduce artifacts along the shadow boundaries, as shown in Figure 9(f). He et al [HXZC21] rely on the facial feature prior for shadow removal, resulting in suboptimal performance due to insensitivity to environmental illumination, as shown in Figure 9(g).…”

Section: Methodsmentioning

confidence: 99%

Facial Image Shadow Removal via Graph‐based Feature Fusion

Zhang,

Chen,

Liu

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Despite natural image shadow removal methods have made significant progress, they often perform poorly for facial image due to the unique features of the face. Moreover, most learning‐based methods are designed based on pixel‐level strategies, ignoring the global contextual relationship in the image. In this paper, we propose a graph‐based feature fusion network (GraphFFNet) for facial image shadow removal. We apply a graph‐based convolution encoder (GCEncoder) to extract global contextual relationships between regions in the coarse shadow‐less image produced by an image flipper. Then, we introduce a feature modulation module to fuse the global topological relation onto the image features, enhancing the feature representation of the network. Finally, the fusion decoder integrates all the effective features to reconstruct the image features, producing a satisfactory shadow‐removal result. Experimental results demonstrate the superiority of the proposed GraphFFNet over the state‐of‐the‐art and validate the effectiveness of facial image shadow removal.

show abstract

“…Our method is compared with existing methods including Yang [26], Guo [23], Gong [25], DeShadowNet [1], STC-GAN [2], DSC [4], Mask-ShadowGAN [9], RIS-GAN [5], DHAN [6], SID [3], LG-shadow [10], G2R [11], SG-ShadNet [12], Auto-exp [13], Bejective [22], SpA-Former [49]. We adopt RMSE, SSIM and PSNR in the LAB color space as evaluation metrics.…”

Section: Experiments a Dataset And Performance Comparisons With Exist...mentioning

confidence: 99%

Exploiting Residual and Illumination with GANs for Shadow Detection and Shadow Removal

Zhang

Long²,

Zhang

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Residual image and illumination estimation have been proven to be helpful for image enhancement. In this paper, we propose a general framework, called RI-GAN, that exploits residual and illumination using generative adversarial networks (GANs). The proposed framework detects and removes shadows in a coarse-to-fine fashion. At the coarse stage, we employ three generators to produce a coarse shadow-removal result, a residual image, and an inverse illumination map. We also incorporate two indirect shadow-removal images via the residual image and the inverse illumination map. With the residual image, the illumination map, and the two indirect shadow-removal images as auxiliary information, the refinement stage estimates a shadow mask to identify shadow regions in the image, and then refines the coarse shadow-removal result to the fine shadow-free image. We introduce a cross-encoding module to the refinement generator, in which the use of feature-crossing can provide additional details to promote the shadow mask and the high-quality shadow-removal result. In addition, we apply data augmentation to the discriminator to reduce the dependence between representations of the discriminator and the quality of the predicted image. Experiments for shadow detection and shadow removal demonstrate that our method outperforms state-of-the-art methods. Furthermore, RI-GAN exhibits good performance in terms of image dehazing, rain removal, and highlight removal, demonstrating the effectiveness and flexibility of the proposed framework.

show abstract

Style-Guided Shadow Removal

Cited by 17 publications

References 42 publications

Structure-Informed Shadow Removal Networks

Structure-Informed Shadow Removal Networks

Facial Image Shadow Removal via Graph‐based Feature Fusion

Exploiting Residual and Illumination with GANs for Shadow Detection and Shadow Removal

Contact Info

Product

Resources

About