Deep learning approach for creating the natural vergence-accommodation conditions in virtual and mixed reality systems

Abstract

A study was made of the causes of the vergence-accommodation conflict of human vision in virtual and mixed reality systems. The technical and algorithmic approaches to reduce and eliminate the vergence-accommodation conflict in virtual reality systems are considered. As a technical solution, an approach was chosen that provides adaptive focusing of the eyepiece of the virtual reality system to the point of convergence of the human eyes, determined by the tracking system of his pupils. Possible algorithmic solutions are considered that provide focusing of the virtual reality image in accordance with the expected accommodation of human eyes. The main solutions are the classical solution of image filtering in accordance with the defocusing caused by natural accommodation at a given distance, and the solution in which the corresponding filtering is performed using neural network technologies. The advantages and disadvantages of the proposed solutions are considered. As a criterion of correctness, we used a visual comparison of the results of image defocusing with the solution obtained by the method of physically correct rendering using the human eye model. As a basis for physically correct rendering, the method of bidirectional stochastic ray tracing with backward photon maps was used. The paper presents an analysis of the advantages and disadvantages of the proposed solutions.