GANPOP: Generative Adversarial Network Prediction of Optical Properties From Single Snapshot Wide-Field Images

Abstract

We present a deep learning framework for widefield, content-aware estimation of absorption and scattering coefficients of tissues, called Generative Adversarial Network Prediction of Optical Properties (GANPOP). Spatial frequency domain imaging is used to obtain ground-truth optical properties from in vivo human hands, freshly resected human esophagectomy samples and homogeneous tissue phantoms. Images of objects with either flat-field or structured illumination are paired with registered optical property maps and are used to train conditional generative adversarial networks that estimate optical properties from a single input image. We benchmark this approach by comparing GANPOP to a single-snapshot optical property (SSOP) technique, using a normalized mean absolute error (NMAE) metric. In human gastrointestinal specimens, GANPOP estimates both reduced scattering and absorption coefficients at 660 nm from a single 0.2 mm -1 spatial frequency illumination image with 58% higher accuracy than SSOP. When applied to both in vivo and ex vivo swine tissues, a GANPOP model trained solely on human specimens and phantoms estimates optical properties with approximately 43% improvement over SSOP, indicating adaptability to sample variety. Moreover, we demonstrate that GANPOP estimates optical properties from flat-field illumination images with similar error to SSOP, which requires structuredillumination. Given a training set that appropriately spans the target domain, GANPOP has the potential to enable rapid and accurate wide-field measurements of optical properties, even from conventional imaging systems with flat-field illumination.Index Terms-optical imaging, tissue optical properties, neural networks, machine learning, spatial frequency domain imaging arXiv:1906.05360v2 [eess.IV]

Rapid tissue oxygenation mapping from snapshot structured-light images with adversarial deep learning

“…This is expected for several reasons. First, the training set used in this study is smaller than in the original GANPOP paper, 28 excluding in vivo hands and tissue-mimicking phantoms. Second, for physical model-based techniques, such as SSOP, the optical property errors due to surface topography variation are correlated across wavelengths and can later be reduced by chromophore fitting.…”

“…The architecture of OxyGAN is based on the GANPOP framework. 28 The generator combines the features of both the U-Net and the ResNet, in that it incorporates both short and long skip connections and is fully residual. As discussed in Ref.…”

“…However, OxyGAN also includes short skip connections within each level and replaces U-Net concatenation with additions, making the network fully residual. 28,37 The residual blocks on each level consist of five 3 × 3 convolutional layers, with residual additions between the outputs of the first and the fourth layers. Dimension reduction on the encoder side and expansion on the decoder side is achieved with 2 × 2 maxpooling and 3 × 3 up-convolutions, respectively.…”

“…The full algorithm was implemented using Pytorch 1.0 on Ubuntu 16.04 with a single NVIDIA Tesla P100 GPU on Google Cloud. 28…”

“…Recently, GANs have been employed to predict optical properties from single structured-illumination images (GANPOP). 28 As a contentaware network, this technique significantly improves upon the accuracy of model-based singlesnapshot techniques in estimating optical properties. However, to compute StO 2 with the GANPOP approach, multiple wavelength-specific networks must be run to first estimate absorption coefficients, followed by chromophore fitting, which compounds errors and increases computational demand [ Fig.…”