The relevance of this topic is due to the rapid development of virtual and augmented reality systems. The problem lies in the formation of natural conditions for lighting objects of the virtual world in real space. To solve a light sources determination problem and recovering its optical parameters were proposed the fully-convolutional neural network, which allows catching the 'behavior of light' features. The output of FCNN is a segmented image with light levels and its strength. Naturally, the fully-convolutional neural network is well suited for image segmentation, so as an encoder was taken the architecture of VGG-16 with layers that pools and convolves an input image to 1x1 pixel and wisely classifies it to one of a class which characterizes its strength. Neural network training was conducted on 221 train images and 39 validation images with learning rate 1E-2 and 200 epochs, after training the loss was 0,2. As a test was used an ‘intersection over union’ method, that compares the ground truth area of an input image and output image, comparing its pixels and giving the result of accuracy. The mean IoU is 0.7, almost rightly classifying the first class with a value of 90 percents of accordance and the last class with a probability of 30 percents.
One of the main problems of mixed reality devices is the physically correct representation of the brightness distribution for virtual objects and their shadows in the real world. In other words, restoring the correct distribution of scene brightness is one of the key parameters to solve the problem of correct interaction between the virtual and real worlds, but neural networks do not allow to determine the position of light sources that are not in line of sight. The paper proposes a method for restoring the parameters of light sources based on the analysis of shadows cast by objects. The results of the proposed method are presented, the accuracy of restoring the position of light sources is estimated and the visual difference between the image of the scene with the original light sources from the same scene with the restored parameters of light sources is demonstrated.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.