Learned hardware-in-the-loop phase retrieval for holographic near-eye displays

Abstract

Holography is arguably the most promising technology to provide wide field-of-view compact eyeglasses-style near-eye displays for augmented and virtual reality. However, the image quality of existing holographic displays is far from that of current generation conventional displays, effectively making today's holographic display systems impractical. This gap stems predominantly from the severe deviations in the idealized approximations of the "unknown" light transport model in a real holographic display, used for computing holograms. In this work, we depart from such approximate "ideal" coherent light transport models for computing holograms. Instead, we learn the deviations of the real display from the ideal light transport from the images measured using a display-camera hardware system. After this unknown light propagation is learned, we use it to compensate for severe aberrations in real holographic imagery. The proposed hardware-in-the-loop approach is robust to spatial, temporal and hardware deviations, and improves the image quality of existing methods qualitatively and quantitatively in SNR and perceptual quality. We validate our approach on a holographic display prototype and show that the method can fully compensate unknown aberrations and erroneous and non-linear SLM phase delays, without explicitly modeling them. As a result, the proposed method significantly outperforms existing state-of-the-art methods in simulation and experimentation - just by observing captured holographic images.

High-contrast, speckle-free, true 3D holography via binary CGH optimization

“…The reproducibility of the speckle-free holographic displays is an important issue due to the sensitiveness of the coherent imaging. Peng et al and Chakravarthula et al deal with the speckle noise generated in the mismatch between the ideal and the actual wave propagation and summarizing the difficulty in reproduction for several SP holographic displays 6 , 32 . The reproducibility depends on the strategy chosen to alleviate the speckle noise.…”

Speckle Noise Suppression Algorithm of Holographic Display Based on Spatial Light Modulator

“…The loss function in the training period consists of two types of loss functions: one is to measure the error of the predicted hologram, and the other one is to measure the quality of the reconstructed 3D object. Through the training strategy, the CNN can learn the characteristic of the hologram with high precision, so as to eliminate speckle noise effectively (Chakravarthula et al, 2020;Peng et al, 2020;Choi et al, 2021b;Wu et al, 2021b).…”

Metameric Varifocal Holography

“…From the recent past, the work by Chakravarthula et al [9] revisits Wirtinger complex derivatives and shows that the visual quality of two-dimensional image reconstructions in CGH can be improved in common Gerchberg-Saxton and Double-phase coding [23] approaches. The works by Peng et al [38] and Chakravarthula et al [10] help to bridge the gap between CGH simulations and actual image reconstructions in a physical display by learning a model of display hardware using a camera and convolutional neural networks. Their findings have drastically improved the quality of two-dimensional image reconstructions in actual holographic displays.…”