Differentiable Microscopy Designs an All Optical Quantitative Phase Microscope

Abstract

Ever since the first microscope by Zacharias Janssen in the late 16th century, scientists have been inventing new types of microscopes for various tasks. Inventing a novel architecture demands years, if not decades, worth of scientific experience and creativity. In this work, we introduce Differentiable Microscopy (∂μ), a deep learning-based design paradigm, to aid scientists design new interpretable microscope architectures. Differentiable microscopy first models a common physics-based optical system however with trainable optical elements at key locations on the optical path. Using pre-acquired data, we then train the model end-to-end for a task of interest. The learnt design proposal can then be simplified by interpreting the learnt optical elements. As a first demonstration, based on the optical 4-f system, we present an all-optical quantitative phase microscope (QPM) design that requires no computational post-reconstruction. A follow-up literature survey suggested that the learnt architecture is similar to the generalized phase concept developed two decades ago. We then incorporate the generalized phase contrast concept to simplify the learning procedure. Furthermore, we also demonstrate that similar results can be achieved by an uninterpretable learning based method, namely diffractive deep neural networks (D2NN). We outperform the existing benchmark for all-optical phase-to-intensity conversion on multiple datasets, and ours is the first demonstration of its kind on D2NNs. The proposed differentiable microscopy framework supplements the creative process of designing new optical systems and would perhaps lead to unconventional but better optical designs.

Differentiable Imaging: A New Tool for Computational Optical Imaging

“…[11] Differentiable programming has been applied to various research fields, including imaging, where it has been successfully used to overcome challenges in current computational imaging techniques. Examples of such applications include ptychography, [12] X-Ray tomography, [13] optical microscopy, [14,15] electron microscopy for denoising and phase imaging. [16,17] In computational imaging, the full potential of differentiable programming has yet to be fully realized.…”

“…[ 86 ] Differentiable microscopy, which employs implements trainable optical elements at key locations on the optical path via differentiable programming, has the potential to create new interpretable microscope architectures or new optical systems, leading to unconventional and better optical designs. [ 15 ]…”