Computational imaging without a computer: seeing through random diffusers at the speed of light

Abstract

Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers. Here, we present a computer-free, all-optical image reconstruction method to see through random diffusers at the speed of light. Using deep learning, a set of transmissive diffractive surfaces are trained to all-optically reconstruct images of arbitrary objects that are completely covered by unknown, random phase diffusers. After the training stage, which is a one-time effort, the resulting diffractive surfaces are fabricated and form a passive optical network that is physically positioned between the unknown object and the image plane to all-optically reconstruct the object pattern through an unknown, new phase diffuser. We experimentally demonstrated this concept using coherent THz illumination and all-optically reconstructed objects distorted by unknown, random diffusers, never used during training. Unlike digital methods, all-optical diffractive reconstructions do not require power except for the illumination light. This diffractive solution to see through diffusers can be extended to other wavelengths, and might fuel various applications in biomedical imaging, astronomy, atmospheric sciences, oceanography, security, robotics, autonomous vehicles, among many others.

Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network

“…Some of these errors can be mitigated by selecting appropriate fabrication methods, e.g., high-precision lithography, and using less absorptive materials. Moreover, our previous results 23 , 38 , 44 , 49 , 50 showed that some of these uncontrolled physical errors and imperfections did not lead to a significant discrepancy between the experimental and numerical, expected results, indicating the correctness of the assumptions involved in our optical forward model and training procedures. Even if these errors and imperfections become considerable, the performance degradation of a diffractive network caused by some of these experimental factors can be compensated by incorporating them as random variables into the physical forward model of the diffractive network during the training process.…”

“…Motivated by the massive success of artificial intelligence and deep learning, in specific, a myriad of new hardware designs for optical computing have been reported recently, including, e.g., on-chip integrated photonic circuits 16 – 22 , free-space optical platforms 23 – 28 , and others 29 – 31 . Among these different optical computing systems, the integration of successive transmissive diffractive layers (forming an optical network) has been demonstrated for optical information processing, such as object classification 23 , 32 – 43 , image reconstruction 38 , 44 , all-optical phase recovery and quantitative phase imaging 45 , and logic operations 46 – 48 . A diffractive network is trained using deep learning and error-backpropagation methods implemented in a digital computer, after which the resulting transmissive layers are fabricated to form a physical network that computes based on the diffraction of the input light through these spatially-engineered transmissive layers.…”

“…It is also scalable since an increase in the input field-of-view (FOV) can be handled by fabricating larger transmissive layers and/or deeper diffractive designs with more successive layers positioned one after another. Furthermore, both the phase and the amplitude information channels of the input scene/FOV can be processed by a diffractive optical network, without the need for phase retrieval or digitizing, vectorizing an image of the scene, which makes diffractive computing highly desirable for machine vision applications 38 , 44 . Harnessing light-matter interactions using engineered diffractive surfaces also enabled the inverse design of optical elements for e.g., spatially-controlled wavelength demultiplexing 49 , pulse engineering 50 , and orbital angular momentum multiplexing/demultiplexing 51 , 52 .…”

Single-layer spatial analog meta-processor for imaging processing

“…In stark contrast, spatial analog computing modulates incident wavefronts in real space, enabling massive and high-throughput parallel operations for required signal-processing tasks such as spatial differentiation 8 – 10 , integration 11 and solving equations 12 , 13 . Conventional physical architectures of spatial domain analog computers leverage upon the phase accumulation with stacked or series of optical elements 14 , 15 , making the whole system bulky and lossy. Nevertheless, metasurface has been captivated as a popular notion and a promising candidate for highly efficient, compact and ultrathin analog processors 16 – 18 .…”

Computer-free computational imaging: optical computing for seeing through random media

“…By combining novel deep neural network (DNN) architectures and domain knowledge in optical physics, the performance limits in various systems are continuously being re-defined, including spatial resolution 3 , 4 , depth-of-field 5 , space-bandwidth product 6 , imaging speed 6 , 7 , sensitivity to low-photon count 8 , and resilience to random scattering 9 , 10 . Of particular interest by Luo et al 11 is the ability to overcome random scattering by a DNN.…”