Deepfakes in Ophthalmology

Chen, Jimmy; Coyner, Aaron S.; Chan, Robison Vernon Paul; Hartnett, M. Elizabeth; Moshfeghi, Darius M.; Owen, Leah A.; Kalpathy–Cramer, Jayashree; Chiang, Michael F.; Campbell, J. Peter

doi:10.1016/j.xops.2021.100079

Cited by 26 publications

(13 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10 In ophthalmic studies, GAN models have been demonstrated in segmentation, [11][12][13][14] data augmentation, [15][16][17][18][19] domain transfer, [20][21][22] image enhancement, [23][24][25][26] and others. 27 In data augmentation with GANs, prior studies on synthetic fundus 15,28,29 and optical coherence tomography (OCT) image generation 16,30 have largely focused on retinal disorders, which are assessed by the presence of lesions. However, studies on the use of GANs for image synthesis in glaucoma, a disease of the optic nerve characterized by progressive loss, have been limited to fundus photography 18 and anterior segment OCT imaging 17 and has not been reported on optic nerve head OCT images (eTable 3 in the Supplement), in which assessment of nerve fiber layer thinning is routinely performed in glaucoma assessment.…”

mentioning

confidence: 99%

Evaluation of Generative Adversarial Networks for High-Resolution Synthetic Image Generation of Circumpapillary Optical Coherence Tomography Images for Glaucoma

et al. 2022

View full text Add to dashboard Cite

IMPORTANCE Deep learning (DL) networks require large data sets for training, which can be challenging to collect clinically. Generative models could be used to generate large numbers of synthetic optical coherence tomography (OCT) images to train such DL networks for glaucoma detection.OBJECTIVE To assess whether generative models can synthesize circumpapillary optic nerve head OCT images of normal and glaucomatous eyes and determine the usability of synthetic images for training DL models for glaucoma detection.DESIGN, SETTING, AND PARTICIPANTS Progressively growing generative adversarial network models were trained to generate circumpapillary OCT scans. Image gradeability and authenticity were evaluated on a clinical set of 100 real and 100 synthetic images by 2 clinical experts. DL networks for glaucoma detection were trained with real or synthetic images and evaluated on independent internal and external test data sets of 140 and 300 real images, respectively.MAIN OUTCOMES AND MEASURES Evaluations of the clinical set between the experts were compared. Glaucoma detection performance of the DL networks was assessed using area under the curve (AUC) analysis. Class activation maps provided visualizations of the regions contributing to the respective classifications.RESULTS A total of 990 normal and 862 glaucomatous eyes were analyzed. Evaluations of the clinical set were similar for gradeability (expert 1: 92.0%; expert 2: 93.0%) and authenticity (expert 1: 51.8%; expert 2: 51.3%). The best-performing DL network trained on synthetic images had AUC scores of 0.97 (95% CI, 0.95-0.99) on the internal test data set and 0.90 (95% CI, 0.87-0.93) on the external test data set, compared with AUCs of 0.96 (95% CI, 0.94-0.99) on the internal test data set and 0.84 (95% CI, 0.80-0.87) on the external test data set for the network trained with real images. An increase in the AUC for the synthetic DL network was observed with the use of larger synthetic data set sizes. Class activation maps showed that the regions of the synthetic images contributing to glaucoma detection were generally similar to that of real images.CONCLUSIONS AND RELEVANCE DL networks trained with synthetic OCT images for glaucoma detection were comparable with networks trained with real images. These results suggest potential use of generative models in the training of DL networks and as a means of data sharing across institutions without patient information confidentiality issues.

show abstract

mentioning

confidence: 99%

Evaluation of Generative Adversarial Networks for High-Resolution Synthetic Image Generation of Circumpapillary Optical Coherence Tomography Images for Glaucoma

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Image-to-image translation has garnered significant attention within the domain of fundus multimodal imaging systems 16 , 22 , 23 . The generation of realistic ICGA images from CF images posed a significant challenge due to the masking effect of the retinal pigment epithelium and the intricate anatomical structure of the choroid.…”

Section: Discussionmentioning

confidence: 99%

Translating color fundus photography to indocyanine green angiography using deep-learning for age-related macular degeneration screening

Chen,

Zhang,

Song

et al. 2024

npj Digit. Med.

View full text Add to dashboard Cite

Age-related macular degeneration (AMD) is the leading cause of central vision impairment among the elderly. Effective and accurate AMD screening tools are urgently needed. Indocyanine green angiography (ICGA) is a well-established technique for detecting chorioretinal diseases, but its invasive nature and potential risks impede its routine clinical application. Here, we innovatively developed a deep-learning model capable of generating realistic ICGA images from color fundus photography (CF) using generative adversarial networks (GANs) and evaluated its performance in AMD classification. The model was developed with 99,002 CF-ICGA pairs from a tertiary center. The quality of the generated ICGA images underwent objective evaluation using mean absolute error (MAE), peak signal-to-noise ratio (PSNR), structural similarity measures (SSIM), etc., and subjective evaluation by two experienced ophthalmologists. The model generated realistic early, mid and late-phase ICGA images, with SSIM spanned from 0.57 to 0.65. The subjective quality scores ranged from 1.46 to 2.74 on the five-point scale (1 refers to the real ICGA image quality, Kappa 0.79–0.84). Moreover, we assessed the application of translated ICGA images in AMD screening on an external dataset (n = 13887) by calculating area under the ROC curve (AUC) in classifying AMD. Combining generated ICGA with real CF images improved the accuracy of AMD classification with AUC increased from 0.93 to 0.97 (P < 0.001). These results suggested that CF-to-ICGA translation can serve as a cross-modal data augmentation method to address the data hunger often encountered in deep-learning research, and as a promising add-on for population-based AMD screening. Real-world validation is warranted before clinical usage.

show abstract

“…Synthetically DL-generated images can be manipulated to adjust image quality and increase diversity of datasets, overcoming implications for privacy laws and data sharing. 26,27 Generative DL models have been recently deployed to reduce image variability across OCT devices, resulting in successful improvement of image quality from time-domain OCT scans to approach that of SDOCT, 28 and in a reduction in the differences among SDOCT images derived from different vendors. 29 However, there is a scarcity of studies specifically targeting the translation of SDOCT to SSOCT and the application of generative DL models to enhance choroidal visibility is an area that remained unexplored.…”

Section: Discussionmentioning

confidence: 99%

Optical Coherence Tomography Choroidal Enhancement Using Generative Deep Learning

Schmetterer,

Bellemo,

Das

et al. 2023

Preprint

View full text Add to dashboard Cite

Spectral domain OCT is the gold standard of imaging the eye in clinics. Penetration depth with such devices is, however, limited and visualization of the choroid, which is essential for diagnosing chorioretinal disease, remains limited. Whereas swept-source OCT (SSOCT) devices allow for visualization of the choroid these instruments are expensive and availability in praxis is limited. We present an artificial intelligence (AI)-based solution to enhance the visualization of the choroid in OCT scans and allow for quantitative measurements of choroidal metrics using generative deep learning (DL). Synthetically enhanced spectral-domain OCT (SDOCT) B-scans with improved choroidal visibility were generated, leveraging matching images to learn deep anatomical features during the training. Using a single-center tertiary eye care institution cohort comprising a total of 362 SDOCT-SSOCT paired subjects, we trained our model with 150,784 images from 410 healthy, 192 glaucoma, and 133 diabetic retinopathy eyes. An independent external test dataset of 37,376 images from 146 eyes was deployed to assess the authenticity and quality of the synthetically enhanced SDOCT images. Experts' ability to differentiate real versus synthetic images was poor (47.5% accuracy). Measurements of choroidal thickness, area, volume, and vascularity index, from the reference SSOCT and synthetically enhanced SDOCT, showed high Pearson's correlations of 0.97 [95% CI: 0.96–0.98], 0.97 [0.95–0.98], 0.95 [0.92–0.98], and 0.87 [0.83–0.91], with intra-class correlation values of 0.99 [0.98–0.99], 0.98 [0.98–0.99], and 0.95 [0.96–0.98], 0.93 [0.91–0.95] respectively. Thus, our DL generative model successfully generated realistic enhanced SDOCT data that is indistinguishable from SSOCT images providing improved visualization of the choroid. This technology enabled accurate measurements of choroidal metrics previously limited by the imaging depth constraints of SDOCT. The findings open new possibilities for utilizing affordable SDOCT devices in studying the choroid in both healthy and pathological conditions.

show abstract

Deepfakes in Ophthalmology

Cited by 26 publications

References 65 publications

Evaluation of Generative Adversarial Networks for High-Resolution Synthetic Image Generation of Circumpapillary Optical Coherence Tomography Images for Glaucoma

Evaluation of Generative Adversarial Networks for High-Resolution Synthetic Image Generation of Circumpapillary Optical Coherence Tomography Images for Glaucoma

Translating color fundus photography to indocyanine green angiography using deep-learning for age-related macular degeneration screening

Optical Coherence Tomography Choroidal Enhancement Using Generative Deep Learning

Contact Info

Product

Resources

About