From picture to 3D hologram: end-to-end learning of real-time 3D photorealistic hologram generation from 2D image input

Abstract

In this Letter, we demonstrate a deep-learning-based method capable of synthesizing a photorealistic 3D hologram in real-time directly from the input of a single 2D image. We design a fully automatic pipeline to create large-scale datasets by converting any collection of real-life images into pairs of 2D images and corresponding 3D holograms and train our convolutional neural network (CNN) end-to-end in a supervised way. Our method is extremely computation-efficient and memory-efficient for 3D hologram generation merely from the knowledge of on-hand 2D image content. We experimentally demonstrate speckle-free and photorealistic holographic 3D displays from a variety of scene images, opening up a way of creating real-time 3D holography from everyday pictures. © 2023 Optical Society of America

Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network

“…For the hologram generation, 3D holography has made remarkable progress in high fidelity and rapid reconstruction of scenes in recent years with the development of neural network technology [30][31][32][33] . For example, the TensorHolo network is proposed to realize the rapid generation of holograms on smart phones 34 .…”

Dimension Expansion Virtual Reality – Head Mounted Display

“…1, we first perform the MiDAS depth estimation process (ver. 3.1 ; dpt_beit_large_512 [5][6]) of one photo and obtain depth estimation layer images. Here, the image shows the farfield when the degree of orange color becomes deeper and deeper.…”