LightNet: Deep Learning Based Illumination Estimation from Virtual Images

Nathan, Sabari; Beham, M. Parisa

doi:10.1007/978-3-030-67070-2_34

Cited by 5 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DeepLight achieves a higher accuracy for the estimated light direction than the approach of Gardner et al [71] by calculating the angular error to the ground-truth light direction and by rendering virtual objects in AR using an estimated light source. LightNet [166] is the latest illumination estimation using a dense network (DenseNet) architecture, which trained with two softmax outputs for colour temperature and lighting direction prediction.…”

Section: For Adaptive Uismentioning

confidence: 99%

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Cao,

Lam,

Lee

et al. 2021

Preprint

View full text Add to dashboard Cite

Mobile Augmented Reality (MAR) integrates computer-generated virtual objects with physical environments for mobile devices. MAR systems enable users to interact with MAR devices, such as smartphones and headworn wearables, and performs seamless transitions from the physical world to a mixed world with digital entities. These MAR systems support user experiences by using MAR devices to provide universal accessibility to digital contents. Over the past 20 years, a number of MAR systems have been developed, however, the studies and design of MAR frameworks have not yet been systematically reviewed from the perspective of user-centric design. This article presents the first effort of surveying existing MAR frameworks (count: 37) and further discusses the latest studies on MAR through a top-down approach: (1) MAR applications; (2) MAR visualisation techniques adaptive to user mobility and contexts; (3) systematic evaluation of MAR frameworks including supported platforms and corresponding features such as tracking, feature extraction plus sensing capabilities; and (4) underlying machine learning approaches supporting intelligent operations within MAR systems. Finally, we summarise the development of emerging research fields, current state-of-the-art, and discuss the important open challenges and possible theoretical and technical directions. This survey aims to benefit both researchers and MAR system developers alike. CCS Concepts: • Human-centered computing → Mixed / augmented reality; Graphical user interfaces; • Software and its engineering → Development frameworks and environments; • Computing methodologies → Machine learning approaches.

show abstract

Section: For Adaptive Uismentioning

confidence: 99%

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Cao,

Lam,

Lee

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Hertz tackle the problem using a multi-scale hierarchical network, the image is encoded at multiple resolutions and feature information is transferred from lower to higher levels to obtain the final transformation. Lastly, Image Lab [35] build on the multilevel hyper vision net [14], adding convolution block attention [52] in their skip connections. Further details of each of these submitted solutions can be found in the supplementary material.…”

Section: Other Submitted Solutionsmentioning

confidence: 99%

AIM 2020: Scene Relighting and Illumination Estimation Challenge

Helou¹,

Zhou²,

Süsstrunk³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We review the AIM 2020 challenge on virtual image relighting and illumination estimation. This paper presents the novel VIDIT dataset used in the challenge and the different proposed solutions and final evaluation results over the 3 challenge tracks. The first track considered one-to-one relighting; the objective was to relight an input photo of a scene with a different color temperature and illuminant orientation (i.e., light source position). The goal of the second track was to estimate illumination settings, namely the color temperature and orientation, from a given image. Lastly, the third track dealt with any-to-any relighting, thus a generalization of the first track. The target color temperature and orientation, rather than being pre-determined, are instead given by a guide image. Participants were allowed to make use of their track 1 and 2 solutions for track 3. The tracks had 94, 52, and 56 registered participants, respectively, leading to 20 confirmed submissions in the final competition stage.

show abstract

“…Moreover, it is easy to generalize the results, and no human intervention is necessary once the model has been trained. Recently, some deep-learning-based methods [ 1 , 7 , 8 , 9 , 10 , 11 ] have been proposed without explicit inverse rendering steps for estimating the scene properties.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, it is easy to generalize the results, and no human intervention is necessary once the model has been trained. Recently, some deep-learning-based methods [1,[7][8][9][10][11] have been proposed without explicit inverse rendering steps for estimating the scene properties. However, deep-learning-based methods suffer from substantial data requirements or complex and high-cost computations, making them unsuitable for real-time applications.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Depth-Guided Relighting Using Lightweight Deep Learning-Based Method

Nathan,

Kansal

2023

J. Imaging

Self Cite

View full text Add to dashboard Cite

Image relighting, which involves modifying the lighting conditions while preserving the visual content, is fundamental to computer vision. This study introduced a bi-modal lightweight deep learning model for depth-guided relighting. The model utilizes the Res2Net Squeezed block’s ability to capture long-range dependencies and to enhance feature representation for both the input image and its corresponding depth map. The proposed model adopts an encoder–decoder structure with Res2Net Squeezed blocks integrated at each stage of encoding and decoding. The model was trained and evaluated on the VIDIT dataset, which consists of 300 triplets of images. Each triplet contains the input image, its corresponding depth map, and the relit image under diverse lighting conditions, such as different illuminant angles and color temperatures. The enhanced feature representation and improved information flow within the Res2Net Squeezed blocks enable the model to handle complex lighting variations and generate realistic relit images. The experimental results demonstrated the proposed approach’s effectiveness in relighting accuracy, measured by metrics such as the PSNR, SSIM, and visual quality.

show abstract

LightNet: Deep Learning Based Illumination Estimation from Virtual Images

Cited by 5 publications

References 21 publications

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

AIM 2020: Scene Relighting and Illumination Estimation Challenge

End-to-End Depth-Guided Relighting Using Lightweight Deep Learning-Based Method

Contact Info

Product

Resources

About