Deep Relighting Networks for Image Light Source Manipulation

Wang, Liwen; Siu, Wan-Chi; Liu, Zhisong; Li, Chu-Tak; Lun, Daniel Pak-Kong

doi:10.1007/978-3-030-67070-2_33

Cited by 25 publications

(22 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have also compared our performance of the model with other two similar Multi-Scale models, Scale-Recurrent Network (SRN) [11] and Dense GridNet [6]. We also compared our work with DRN [12] as they had worked on the same problem with same dataset. For qualitative comparison against other networks, example of output images are shown in Fig.…”

Section: Resultsmentioning

confidence: 99%

Multiple Scale One-to-One Relighting Network

Yang¹

2021

Preprint

View full text Add to dashboard Cite

Significant picture relighting grants photo change by illumination-specific altering without human effort and so it is getting much interested of late. Most of the existing predominant procedures open for relighting are run-time genuinely and memory inefficient. Keeping these issues in judgment skills, we propose the utilize of Stacked Significant Multi-Scale Dynamic Organize, which sums highlights from each picture at different scales. Our course of action is differentiable and solid for decoding picture brightening setting from input picture to target picture. Besides, we have besides showed up that utilizing a multi-step planning approach to this issue with two unmistakable hardship capacities can inside and out boost execution and can finish a tall quality diversion of a relighted picture.

show abstract

Section: Resultsmentioning

confidence: 99%

Multiple Scale One-to-One Relighting Network

Yang¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Encoder-decoder structures can be employed to generate images. For example, in [32], a two-way encoder-decoder structure generates relighted photos that come with lighting with desired direction and color temperature. It is trained in a supervised fashion to generate an intermediate shadow-free image and prior image of the desired lighting and to combine them into a final relighted output.…”

Section: Related Workmentioning

confidence: 99%

Shadow Estimation for Ultrasound Images Using Auto-Encoding Structures and Synthetic Shadows

et al. 2021

View full text Add to dashboard Cite

Acoustic shadows are common artifacts in medical ultrasound imaging. The shadows are caused by objects that reflect ultrasound such as bones, and they are shown as dark areas in ultrasound images. Detecting such shadows is crucial for assessing the quality of images. This will be a pre-processing for further image processing or recognition aiming computer-aided diagnosis. In this paper, we propose an auto-encoding structure that estimates the shadowed areas and their intensities. The model once splits an input image into an estimated shadow image and an estimated shadow-free image through its encoder and decoder. Then, it combines them to reconstruct the input. By generating plausible synthetic shadows based on relatively coarse domain-specific knowledge on ultrasound images, we can train the model using unlabeled data. If pixel-level labels of the shadows are available, we also utilize them in a semi-supervised fashion. By experiments on ultrasound images for fetal heart diagnosis, we show that our method achieved 0.720 in the DICE score and outperformed conventional image processing methods and a segmentation method based on deep neural networks. The capability of the proposed method on estimating the intensities of shadows and the shadow-free images is also indicated through the experiments.

show abstract

“…It features indoor and outdoor scenes lit from 8 directions and captured under 5 temperature settings. The applications of VIDIT include [24] where Wang et al tackle the task of relighting in stages. They simultaneously teach two network branches to remove the effects of source illumination and to apply shadows according to target illumination, and then combine them to render the final result.…”

Section: Literature Reviewmentioning

confidence: 99%

SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition

Nikolina¹,

Mustafa²,

Phillipson³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present SILT, a Self-supervised Implicit Lighting Transfer method. Unlike previous research on scene relighting, we do not seek to apply arbitrary new lighting configurations to a given scene. Instead, we wish to transfer the lighting style from a database of other scenes, to provide a uniform lighting style regardless of the input. The solution operates as a two-branch network that first aims to map input images of any arbitrary lighting style to a unified domain, with extra guidance achieved through implicit image decomposition. We then remap this unified input domain using a discriminator that is presented with the generated outputs and the style reference, i.e. images of the desired illumination conditions. Our method is shown to outperform supervised relighting solutions across two different datasets without requiring lighting supervision. The code and pre-trained models can be found here. IntroductionWe propose the problem of lighting transfer where an input image under arbitrary illumination conditions is adapted to match the lighting of a style reference database. Previous approaches to a similar problem -arbitrary image relighting -either make simplifying assumptions about the scene (e.g. a single dominant object or a single light source) or require costly supervision where identical scenes must be captured under a large number of known lighting conditions. In contrast, our lighting transfer approach, SILT, is entirely self-supervised. SILT requires only a training dataset with multiple examples of the same scene and a style reference database. It is not necessary to know the ground truth lighting conditions for the training data, nor is it necessary for the target illumination to be present within the training dataset. This distinction between image relighting and lighting transfer is demonstrated in Fig. 1.The proposed SILT method consists of a two-branch network. During training, the generator sees a number of input images of the same scene under different unknown illuminations. The model attempts to enforce similarity between the illumination conditions of the outputs

show abstract

Deep Relighting Networks for Image Light Source Manipulation

Cited by 25 publications

References 35 publications

Multiple Scale One-to-One Relighting Network

Multiple Scale One-to-One Relighting Network

Shadow Estimation for Ultrasound Images Using Auto-Encoding Structures and Synthetic Shadows

SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition

Contact Info

Product

Resources

About