Super-resolution time-resolved imaging using computational sensor fusion

Abstract

Imaging across both the full transverse spatial and temporal dimensions of a scene with high precision in all three coordinates is key to applications ranging from LIDAR to fluorescence lifetime imaging. However, compromises that sacrifice, for example, spatial resolution at the expense of temporal resolution are often required, in particular when the full 3-dimensional data cube is required in short acquisition times. We introduce a sensor fusion approach that combines data having low-spatial resolution but high temporal precision gathered with a single-photon-avalanche-diode (SPAD) array with data that has high spatial but no temporal resolution, such as that acquired with a standard CMOS camera. Our method, based on blurring the image on the SPAD array and computational sensor fusion, reconstructs time-resolved images at significantly higher spatial resolution than the SPAD input, upsampling numerical data by a factor $$12 \times 12$$ 12 × 12 , and demonstrating up to $$4 \times 4$$ 4 × 4 upsampling of experimental data. We demonstrate the technique for both LIDAR applications and FLIM of fluorescent cancer cells. This technique paves the way to high spatial resolution SPAD imaging or, equivalently, FLIM imaging with conventional microscopes at frame rates accelerated by more than an order of magnitude.

Working at the interface of physics and biology: An early career researcher perspective

“…A statistics-based approach is used to estimate FRET efficiency with high spatial fidelity, by combining time-resolved single-photon quantum detectors with high-resolution cameras. 5 , 6 …”

Single-sample image-fusion upsampling of fluorescence lifetime images

“…Reconstruction-based modeling instead manipulates the detection to optically redistribute information about the high-resolution target into fewer measurements. This encoding provides a mathematical forward model that is used to reconstruct the nonsampled points in an inverse retrieval framework, for example, via point spread function engineering ( 34 ), blurring ( 35 ), or compressed sensing ( 36 – 39 ).…”

Fast Analysis of Time-Domain Fluorescence Lifetime Imaging via Extreme Learning Machine

“…Iterative fitting and optimization approaches were reported to deduce fluorescence lifetimes. A convex optimization method [ 11 ] was utilized for high-resolution FLIM, where the accuracy is related to fine-tuned hyperparameters in the cost function. An F-value-based optimization algorithm [ 12 ] was used to minimize signal distortion introduced by pile-up effects and the dead time of single-photon detectors.…”