Deep learning model for ultrafast multifrequency optical property extractions for spatial frequency domain imaging

Zhao, Yanyu; Deng, Yue; Bao, Feng; Peterson, Hannah; Istfan, Raeef; Roblyer, Darren

doi:10.1364/ol.43.005669

Cited by 48 publications

(39 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite its prevalence and increasing importance in the field of medical imaging, machine learning has only recently been explored for optical property mapping. This includes a random forest regressor to replace the nonlinear model inversion [32], and using deep neural networks to reconstruct optical properties from multifrequency measurements [33]. Both of these approaches aim to bypass the time-consuming LUT step in SFDI.…”

Section: Machine Learning In Optical Property Estimationmentioning

confidence: 99%

GANPOP: Generative Adversarial Network Prediction of Optical Properties From Single Snapshot Wide-Field Images

Chen

Mahmood

Sweer

et al. 2020

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

We present a deep learning framework for widefield, content-aware estimation of absorption and scattering coefficients of tissues, called Generative Adversarial Network Prediction of Optical Properties (GANPOP). Spatial frequency domain imaging is used to obtain ground-truth optical properties from in vivo human hands, freshly resected human esophagectomy samples and homogeneous tissue phantoms. Images of objects with either flat-field or structured illumination are paired with registered optical property maps and are used to train conditional generative adversarial networks that estimate optical properties from a single input image. We benchmark this approach by comparing GANPOP to a single-snapshot optical property (SSOP) technique, using a normalized mean absolute error (NMAE) metric. In human gastrointestinal specimens, GANPOP estimates both reduced scattering and absorption coefficients at 660 nm from a single 0.2 mm -1 spatial frequency illumination image with 58% higher accuracy than SSOP. When applied to both in vivo and ex vivo swine tissues, a GANPOP model trained solely on human specimens and phantoms estimates optical properties with approximately 43% improvement over SSOP, indicating adaptability to sample variety. Moreover, we demonstrate that GANPOP estimates optical properties from flat-field illumination images with similar error to SSOP, which requires structuredillumination. Given a training set that appropriately spans the target domain, GANPOP has the potential to enable rapid and accurate wide-field measurements of optical properties, even from conventional imaging systems with flat-field illumination.Index Terms-optical imaging, tissue optical properties, neural networks, machine learning, spatial frequency domain imaging arXiv:1906.05360v2 [eess.IV]

show abstract

Section: Machine Learning In Optical Property Estimationmentioning

confidence: 99%

GANPOP: Generative Adversarial Network Prediction of Optical Properties From Single Snapshot Wide-Field Images

Chen

Mahmood

Sweer

et al. 2020

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…One potential solution to decrease this delay is to increase the acquisition rate (500 fps for instance) but it requires to increase the light fluence from the sources to maintain proper SNR and prevent noise in the images. Finally, current state-of-the-art processing methods in the spatial frequency domain include optimized LUTs (down to 18 ms for 572 × 672 pixels), 24 as well as deep learning and machine learning methods (down to 200 ms for 696 × 520 pixels), 28,29 and soon, faster methods that will enable real-time processing and visualization of data will certainly be developed. Efforts are ongoing to combine rapid processing of SSOP data, with spatiotemporal modulation of light, onto a clinically compatible device to test and validate the capacity for diffuse optical imaging to provide relevant data in real time during concrete clinical applications.…”

Section: Discussionmentioning

confidence: 99%

Real-time, wide-field, and quantitative oxygenation imaging using spatiotemporal modulation of light

et al. 2019

View full text Add to dashboard Cite

Quantitative diffuse optical imaging has the potential to provide valuable functional information about tissue status, such as oxygen saturation or blood content to healthcare practitioners in real time. However, significant technical challenges have so far prevented such tools from being deployed in the clinic. Toward achieving this goal, prior research introduced methods based on spatial frequency domain imaging (SFDI) that allow real-time (within milliseconds) wide-field imaging of optical properties but at a single wavelength. However, for this technology to be useful to clinicians, images must be displayed in terms of metrics related to the physiological state of the tissue, hence interpretable to guide decision-making. For this purpose, recent developments introduced multispectral SFDI methods for rapid imaging of oxygenation parameters up to 16 frames per seconds (fps). We introduce real-time, wide-field, and quantitative blood parameters imaging using spatiotemporal modulation of light. Using this method, we are able to quantitatively obtain optical properties at 100 fps at two wavelengths (665 and 860 nm), and therefore oxygenation, oxyhemoglobin, and deoxyhemoglobin, using a single camera with, at most, 4.2% error in comparison with standard SFDI acquisitions.

show abstract

“…Multispatial frequency processing codes were developed to take into account this effect in cases where divergence is significant, such as within an endoscope. 16,40 Finally, many other sources of errors such as camera linearity, projection linearity, and source stability, as in any optical design should be taken into account, either by calibration or monitoring and/or appropriate analytic or empirical corrections. 41,42 One aspect that should not be neglected is the quality of the projection and acquisition, and the resulting demodulation as this will have a direct influence onto the precision and accuracy of the measurement.…”

Section: Key Considerationsmentioning

confidence: 99%

“…Multiple approaches have been developed for analyzing SFD signals to determine homogeneous or spatially resolved optical properties, including least-squares methods, 2,43 LUTs, 1,44 machine learning and deep learning, 40,45 and data matching to empirical phantom grids. 46 The choice of computation model is important and application specific.…”

Section: Inverse Models and Data Processingmentioning

confidence: 99%

Spatial frequency domain imaging in 2019: principles, applications, and perspectives

2019

View full text Add to dashboard Cite

Spatial frequency domain imaging (SFDI) has witnessed very rapid growth over the last decade, owing to its unique capabilities for imaging optical properties and chromophores over a large field-of-view and in a rapid manner. We provide a comprehensive review of the principles of this imaging method as of 2019, review the modeling of light propagation in this domain, describe acquisition methods, provide an understanding of the various implementations and their practical limitations, and finally review applications that have been published in the literature. Importantly, we also introduce a group effort by several key actors in the field for the dissemination of SFDI, including publications, advice in hardware and implementations, and processing code, all freely available online.

show abstract

Deep learning model for ultrafast multifrequency optical property extractions for spatial frequency domain imaging

Cited by 48 publications

References 28 publications

GANPOP: Generative Adversarial Network Prediction of Optical Properties From Single Snapshot Wide-Field Images

GANPOP: Generative Adversarial Network Prediction of Optical Properties From Single Snapshot Wide-Field Images

Real-time, wide-field, and quantitative oxygenation imaging using spatiotemporal modulation of light

Spatial frequency domain imaging in 2019: principles, applications, and perspectives

Contact Info

Product

Resources

About