Bijective-constrained cycle-consistent deep learning for optics-free imaging and classification

Abstract

Many deep learning approaches to solve computational imaging problems have proven successful through relying solely on the data. However, when applied to the raw output of a bare (optics-free) image sensor, these methods fail to reconstruct target images that are structurally diverse. In this work we propose a self-consistent supervised model that learns not only the inverse, but also the forward model to better constrain the predictions through encouraging the network to model the ideal bijective imaging system. To do this, we employ cycle consistency alongside traditional reconstruction losses, both of which we show are needed for incoherent optics-free image reconstruction. By eliminating all optics, we demonstrate imaging with the thinnest camera possible.

Perspectives on Imaging with Diffractive Flat Optics

“…Computational imaging has provided examples of nonanthropocentric metrics and associated solutions in the form of PSF-engineered optics, 39,51 as well as single-pixel, 66 lensless, 40,67 and optics-free cameras. 52,68 This dovetails with our previous comment on the importance of co-optimizing hardware with computation, particularly in the context of application-specif ic imaging. These ideas also inevitably blur the lines between imaging and sensing, for instance, when gleaning some information from a scene, i.e., inferencing is the ultimate goal, rather than forming an image that is pleasing to a human brain.…”

“…See ref for an early example in microscopy. Such approaches point one way forward for nonanthropocentric inferencing (rather than imaging), where a machine is attempting to understand its surroundings, which may be possible even in the absence of optics …”

“…Such approaches point one way forward for nonanthropocentric inferencing (rather than imaging), where a machine is attempting to understand its surroundings, which may be possible even in the absence of optics. 52 For (anthropocentric) imaging applications, the strehl ratio is often used as a metric for comparing different lens systems. The strehl ratio is the ratio of the on-axis intensity of the 6.…”

Fast Multichannel Inverse Design through Augmented Partial Factorization

“…Nanoengineered photonic devices can realize versatile and high-performance functionalities in a compact footprint, expanding the limited scope of conventional optical components. Computer-automated inverse design – can search a high-dimensional parameter space to discover optimal structures that outperform manual designs or realize new functionalities. With an inverse design, the photonic structure is updated iteratively to optimize an objective function f that encapsulates the desired properties.…”

EventLFM: event camera integrated Fourier light field microscopy for ultrafast 3D imaging

“…These methods first encode 3D information into 2D multiplexed measurements and then reconstruct the 3D volume computationally. Examples of such techniques include light field microscopy (LFM) [5][6][7][8][9] , lensless imaging [10][11][12] , and point-spread-function (PSF) engineering 13,14 . Conventional LFM works by inserting a microlens array (MLA) into the native image plane of a widefield microscope 5 .…”