Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells

Abstract

Fluorescence lifetime imaging microscopy (FLIM) is a powerful tool to quantify molecular compositions and study molecular states in complex cellular environment as the lifetime readings are not biased by fluorophore concentration or excitation power. However, the current methods to generate FLIM images are either computationally intensive or unreliable when the number of photons acquired at each pixel is low. Here we introduce a new deep learning-based method termed flimGANE (fluorescence lifetime imaging based on Generative Adversarial Network Estimation) that can rapidly generate accurate and high-quality FLIM images even in the photon-starved conditions. We demonstrated our model is up to 2,800 times faster than the gold standard time-domain maximum likelihood estimation (TD_MLE) and that flimGANE provides a more accurate analysis of low-photon-count histograms in barcode identification, cellular structure visualization, Förster resonance energy transfer characterization, and metabolic state analysis in live cells. With its advantages in speed and reliability, flimGANE is particularly useful in fundamental biological research and clinical applications, where high-speed analysis is critical.

Intraoperative In Vivo Imaging Modalities in Head and Neck Cancer Surgical Margin Delineation: A Systematic Review

“…Despite showing great promise, FLIM autofluorescence imaging has been limited by time-consuming, computationally intensive models, and this is particularly true with lower photon/pixel ratios [ 62 ]. However, the field of autofluorescence imaging is rapidly evolving, with newer techniques including DOCI being developed to overcome these challenges with prompt image acquisition.…”

“…This new modality functions at 258-fold the speed of LSE and impressively 2800-fold faster than the gold standard maximum likelihood estimation (MLE) model. Most importantly, flimGANE has been shown to perform consistently with a photon-count-to-pixel ratio of 50, which is half of what is typically required in MLE [ 62 ]. Furthermore, the incorporation of augmented reality with FLIM represents another exciting area in which this field is moving towards, showing promising results in head and neck surgery [ 65 ].…”

Nanozyme-based colorimetric biosensor with a systemic quantification algorithm for noninvasive glucose monitoring

“…However, fluorescence biosensors have limitations, including potential toxicity issues, susceptibility to interference due to pH changes and oxygen concentrations, short lifespan of the fluorophore, photostability issues, loss of recognition capability, and a low signal-to-noise ratio. In addition, conventional statistical algorithms are often limited by low accuracy under low illumination conditions, long computation times, and incorrect initial assumptions of decay parameter 46 . Therefore, machine learning-based simple training architectures of artificial neural network (ANN) or convolutional neural network (CNN) have been employed to improve the visualization, less computational time, and detect low fluorescent signal.…”

Simple and Robust Deep Learning Approach for Fast Fluorescence Lifetime Imaging

“…Another MLP [30] processes phasor coordinates and generates lifetime parameters. Because the inputs are coordinates rather than histograms, the MLP 3-D CNN [31] 1-D CNN [25] 1-D CNN [28] MLP [27] Phasor-MLP [30] MLP [29] GAN [33] ELM [36] FLAN FLAN+LS The model size was not given in the paper; the size was estimated from the network's structure. exhibits fast speed.…”

“…Besides, Ruoyang et al proposed a 3-D CNN combined with CS [32] for wide-field FLIM imaging in vitro and in vivo environments. Further, to address the challenge of FLIM with photon-starved conditions, Yuan-I et al reported a generative adversarial network [33] for FLIM imaging in low-photon scenarios (below 400). Another challenge of FLIM is the spectral overlap of fluorescent emissions leading to bleedthrough.…”