Deep learning-based video stream reconstruction in mass-production diffractive optical systems

Abstract

Many recent studies have focused on developing image reconstruction algorithms in optical systems based on flat optics. These studies demonstrate the feasibility of applying a combination of flat optics and the reconstruction algorithms in real vision systems. However, additional causes of quality loss have been encountered in the development of such systems. This study investigates the influence on the reconstructed image quality of such factors as limitations of mass production technology for diffractive optics, lossy video stream compression artifacts, and specificities of a neural network approach to image reconstruction. The paper offers an end-to-end deep learning-based image reconstruction framework to compensate for the additional factors of quality losing. It provides the image reconstruction quality sufficient for applied vision systems.

Hybrid Refractive-Diffractive Lens with Reduced Chromatic and Geometric Aberrations and Learned Image Reconstruction

“…Most of these methods are based on the known image degradation models and range from simple downsampling with a bicubic upsampling [ 28 , 29 , 30 , 31 , 32 ] to more recent works, relying on blurring kernel degradation [ 30 , 31 , 32 ]. When applied to real-world images, these algorithms suffer from artifacts because the real image degradation is usually too complex [ 33 ] or has a non-local behavior that depends on the image content [ 16 , 34 ]. Artifact-free results can still be achieved with techniques described in [ 33 , 35 , 36 ].…”

“…To build a semi-real dataset for supervised learning, we use a capture-from-screen laboratory setup [ 16 ] with a laptop connected to a UHD LCD monitor and Basler acA1920-40uc USB 3.0 camera with our doublet lens system. However, this setup has three main differences from real scene capturing: a higher dynamic range, camera gain, and lossy video compression.…”

“…A combination of refractive-diffractive lenses with computational image reconstruction allowed us to reach the image quality that real-world applications demand. In previous works [ 16 , 17 , 18 , 19 ], deep learning-based image reconstruction was successfully used to compensate for the chromatic distortions typical in an optical system with a harmonic diffractive lens. Despite good results of the deep learning-based correction as measured by the peak signal-to-noise ratio (PSNR) on a test set at about 27 dB [ 16 , 20 ], reconstructed real scene images showed visible reconstruction artifacts.…”

“…In previous works [ 16 , 17 , 18 , 19 ], deep learning-based image reconstruction was successfully used to compensate for the chromatic distortions typical in an optical system with a harmonic diffractive lens. Despite good results of the deep learning-based correction as measured by the peak signal-to-noise ratio (PSNR) on a test set at about 27 dB [ 16 , 20 ], reconstructed real scene images showed visible reconstruction artifacts. These artifacts were caused by the following features specific to real scenes as opposed to the training set: high dynamic range (HDR), camera gain, and lossy video compression.…”

Improved Fault Diagnosis in Hydraulic Systems with Gated Convolutional Autoencoder and Partially Simulated Data

“…Papers [ 12 , 13 ] consider model-based and DSP-based fault prediction, while papers [ 14 , 15 ] and more recent ones use data-driven approaches. Data-driven and deep learning-based methods show great results not only in computer vision applications [ 16 , 17 ], speech recognition [ 18 ], natural language processing [ 19 ], and medical imaging [ 20 , 21 ], but also as classifiers for induction motor fault classification [ 22 ], railway vehicle wheels diagnosis [ 23 ], industrial machinery [ 24 ], hydraulic system malfunction identification [ 25 ], and fault diagnosis of aircraft engines [ 26 ]. A comprehensive review of intelligent fault diagnostic methods presented as a single timeline is provided in [ 27 ].…”