Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators

Choi, Suyeon; Gopakumar, Manu; Peng, Yifan; Kim, Jong-Hyun; O’Toole, Matthew; Wetzstein, Gordon

doi:10.1145/3528233.3530734

Cited by 45 publications

(36 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, the system can take advantage of advanced features of SLM such as high framerate or complex modulation capability 57 . Several studies have recently reported that utilization of temporal multiplexing helps speckle reduction and image quality enhancement 48,58 . Such method can be utilized to improve the image quality.…”

Section: Discussionmentioning

confidence: 99%

Waveguide Holography: Towards True 3D Holographic Glasses

Jang

Bang

Chae

et al. 2022

Preprint

View full text Add to dashboard Cite

We present a novel near-eye display concept which consists of a waveguide combiner, a spatial light modulator, and a laser light source. The proposed system can display true 3D holographic images through see-through pupil-replicating waveguide combiner as well as providing a large eye-box. By modeling the coherent light interaction inside of the waveguide combiner, we demonstrate that the output wavefront from the waveguide can be controlled by modulating the wavefront of input light using a spatial light modulator. This new possibility allows combining a holographic display, which is considered as the ultimate 3D display technology, with the state-of-the-art pupil replicating waveguides, enabling the path towards true 3D holographic augmented reality glasses.

show abstract

Section: Discussionmentioning

confidence: 99%

Waveguide Holography: Towards True 3D Holographic Glasses

Jang

Bang

Chae

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The reference hologram used as a metric was obtained by the BE-ASM under double sampling because that the BE-ASM has the highest accuracy. In general, images with a PSNR greater than 30 dB are considered acceptable because the human eye has a limited ability to recognize noise [51], and the highest quality of holographic displays in current experiments [24,27] is difficult to reach 30 dB. Therefore, to obtain higher quality holograms, we select 𝜂 = 91% as an energy threshold in all calculations below, with a PSNR of 37.5 dB and SSIM of 0.93.…”

Section: Solving For the Frequency Sampling Boundary Of Ce-asmmentioning

confidence: 99%

“…Alternatively, light field stereogram technique has a clear advantage in implementing full parallax [61][62][63], because it execute occlusion and shading processes for each viewpoint with well refined 3D computer graphics techniques. Recently, light-field-based hologram calculations using deep learning has been proposed to achieve unparalleled image quality [27]. However, light field methods have the problem of requiring re-rendering for each element image, while the proposed polygon-based method is a timely rendering of the raw 3D object within the pipeline.…”

Section: Limitationsmentioning

confidence: 99%

“…However, for a smooth and continuous complex 3D object, the computational effort versus the reconstructed image quality is still a trade-off [22]. Deep learning-generated holograms using layer-based RGBD data open exciting avenues for fewer calculations and higher quality [23][24][25][26][27]. Nevertheless, CGHs using deep learning do not simulate optical diffraction and interference processes, which may cause unwanted artifacts in scenes with a large range depth and hinder the generation of holograms in large fields of view [22].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High-speed rendering pipeline for polygon-based holograms

Wang¹,

Ito²,

Shimobaba³

2023

Photon. Res.

View full text Add to dashboard Cite

As an important three-dimensional (3D) display technology, computer-generated holograms (CGHs) have been facing challenges of computational efficiency and realism. The polygon-based method, as the mainstream CGH algorithm, has been widely studied and improved over the past 20 years. However, few comprehensive and high-speed methods have been proposed. In this study, we propose an analytical spectrum method based on the principle of spectral energy concentration, which can achieve a speedup of nearly 30 times and generate high-resolution (8K) holograms with low memory requirements. Based on the Phong illumination model and the sub-triangles method, we propose a shading rendering algorithm to achieve a very smooth and realistic reconstruction with only a small increase in computational effort. Benefiting from the idea of triangular subdivision and octree structures, the proposed original occlusion culling scheme can closely crop the overlapping areas with almost no additional overhead, thus rendering a 3D parallax sense. With this, we built a comprehensive high-speed rendering pipeline of polygon-based holograms capable of computing any complex 3D object. Numerical and optical reconstructions confirmed the generalizability of the pipeline.

show abstract

“…Timemultiplexed method is used to reduce the effect of the speckles by the temporal superposition of multiple independent speckle patterns. 6,7 Whereas, the time-multiplexed method requires time-sequentially uploading multiple holograms for a target image, which would sacrifice the refresh rates of the SLMs. One alternative method for speckle reduction is to reduce the coherence of the light source.…”

Section: Introductionmentioning

confidence: 99%

Weighted constraint stochastic gradient descent algorithm for computational holographic near-eye display

Chen

Zhu

Tian

et al. 2022

Holography, Diffractive Optics, and Applications XII

View full text Add to dashboard Cite

Stochastic gradient descent (SGD) algorithm with weighted constraint strategy is proposed to solve the vortex stagnation problem in CGH optimization and improve the image quality for computational holographic near-eye display. The weighted constraint strategy includes weighted phase constraint and weighted amplitude constraint. The weighted phase constraint is used to smooth the phase profile of reconstructed field, which helps to solve the vortex stagnation problem caused by optical vortices and eliminate the speckles in reconstructed field. The weighted amplitude constraint is used to broaden the optimization space by introducing the amplitude freedom of non-signal region in the reconstructed field, which helps to further improve the image quality in the signal region. The weighted constraint SGD algorithm can ensure the stable convergence of CGH optimization and avoid the vortex stagnation, which helps to eliminate the speckles and improve the image quality for holographic near-eye display.

show abstract

Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators

Cited by 45 publications

References 21 publications

Waveguide Holography: Towards True 3D Holographic Glasses

Waveguide Holography: Towards True 3D Holographic Glasses

High-speed rendering pipeline for polygon-based holograms

Weighted constraint stochastic gradient descent algorithm for computational holographic near-eye display

Contact Info

Product

Resources

About