Abstract:In vivo confocal microscopy (IVCM) is a noninvasive, reproducible, and inexpensive diagnostic tool for corneal diseases. However, widespread and effortless image acquisition in IVCM creates serious image analysis workloads on ophthalmologists, and neural networks could solve this problem quickly. We have produced a novel deep learning algorithm based on generative adversarial networks (GANs), and we compare its accuracy for automatic segmentation of subbasal nerves in IVCM images with a fully convolutional neu… Show more
“…Recently, generative adversarial networks (GANs) 9 have been extensively used in the artificial intelligence community for the synthesis of artificial images and have emerged as a potential technique to address the challenge of data scarcity in biomedical applications. 10 In ophthalmic studies, GAN models have been demonstrated in segmentation, [11][12][13][14] data augmentation, [15][16][17][18][19] domain transfer, [20][21][22] image enhancement, [23][24][25][26] and others. 27 In data augmentation with GANs, prior studies on synthetic fundus 15,28,29 and optical coherence tomography (OCT) image generation 16,30 have largely focused on retinal disorders, which are assessed by the presence of lesions.…”
IMPORTANCE Deep learning (DL) networks require large data sets for training, which can be challenging to collect clinically. Generative models could be used to generate large numbers of synthetic optical coherence tomography (OCT) images to train such DL networks for glaucoma detection.OBJECTIVE To assess whether generative models can synthesize circumpapillary optic nerve head OCT images of normal and glaucomatous eyes and determine the usability of synthetic images for training DL models for glaucoma detection.DESIGN, SETTING, AND PARTICIPANTS Progressively growing generative adversarial network models were trained to generate circumpapillary OCT scans. Image gradeability and authenticity were evaluated on a clinical set of 100 real and 100 synthetic images by 2 clinical experts. DL networks for glaucoma detection were trained with real or synthetic images and evaluated on independent internal and external test data sets of 140 and 300 real images, respectively.MAIN OUTCOMES AND MEASURES Evaluations of the clinical set between the experts were compared. Glaucoma detection performance of the DL networks was assessed using area under the curve (AUC) analysis. Class activation maps provided visualizations of the regions contributing to the respective classifications.RESULTS A total of 990 normal and 862 glaucomatous eyes were analyzed. Evaluations of the clinical set were similar for gradeability (expert 1: 92.0%; expert 2: 93.0%) and authenticity (expert 1: 51.8%; expert 2: 51.3%). The best-performing DL network trained on synthetic images had AUC scores of 0.97 (95% CI, 0.95-0.99) on the internal test data set and 0.90 (95% CI, 0.87-0.93) on the external test data set, compared with AUCs of 0.96 (95% CI, 0.94-0.99) on the internal test data set and 0.84 (95% CI, 0.80-0.87) on the external test data set for the network trained with real images. An increase in the AUC for the synthetic DL network was observed with the use of larger synthetic data set sizes. Class activation maps showed that the regions of the synthetic images contributing to glaucoma detection were generally similar to that of real images.CONCLUSIONS AND RELEVANCE DL networks trained with synthetic OCT images for glaucoma detection were comparable with networks trained with real images. These results suggest potential use of generative models in the training of DL networks and as a means of data sharing across institutions without patient information confidentiality issues.
“…Recently, generative adversarial networks (GANs) 9 have been extensively used in the artificial intelligence community for the synthesis of artificial images and have emerged as a potential technique to address the challenge of data scarcity in biomedical applications. 10 In ophthalmic studies, GAN models have been demonstrated in segmentation, [11][12][13][14] data augmentation, [15][16][17][18][19] domain transfer, [20][21][22] image enhancement, [23][24][25][26] and others. 27 In data augmentation with GANs, prior studies on synthetic fundus 15,28,29 and optical coherence tomography (OCT) image generation 16,30 have largely focused on retinal disorders, which are assessed by the presence of lesions.…”
IMPORTANCE Deep learning (DL) networks require large data sets for training, which can be challenging to collect clinically. Generative models could be used to generate large numbers of synthetic optical coherence tomography (OCT) images to train such DL networks for glaucoma detection.OBJECTIVE To assess whether generative models can synthesize circumpapillary optic nerve head OCT images of normal and glaucomatous eyes and determine the usability of synthetic images for training DL models for glaucoma detection.DESIGN, SETTING, AND PARTICIPANTS Progressively growing generative adversarial network models were trained to generate circumpapillary OCT scans. Image gradeability and authenticity were evaluated on a clinical set of 100 real and 100 synthetic images by 2 clinical experts. DL networks for glaucoma detection were trained with real or synthetic images and evaluated on independent internal and external test data sets of 140 and 300 real images, respectively.MAIN OUTCOMES AND MEASURES Evaluations of the clinical set between the experts were compared. Glaucoma detection performance of the DL networks was assessed using area under the curve (AUC) analysis. Class activation maps provided visualizations of the regions contributing to the respective classifications.RESULTS A total of 990 normal and 862 glaucomatous eyes were analyzed. Evaluations of the clinical set were similar for gradeability (expert 1: 92.0%; expert 2: 93.0%) and authenticity (expert 1: 51.8%; expert 2: 51.3%). The best-performing DL network trained on synthetic images had AUC scores of 0.97 (95% CI, 0.95-0.99) on the internal test data set and 0.90 (95% CI, 0.87-0.93) on the external test data set, compared with AUCs of 0.96 (95% CI, 0.94-0.99) on the internal test data set and 0.84 (95% CI, 0.80-0.87) on the external test data set for the network trained with real images. An increase in the AUC for the synthetic DL network was observed with the use of larger synthetic data set sizes. Class activation maps showed that the regions of the synthetic images contributing to glaucoma detection were generally similar to that of real images.CONCLUSIONS AND RELEVANCE DL networks trained with synthetic OCT images for glaucoma detection were comparable with networks trained with real images. These results suggest potential use of generative models in the training of DL networks and as a means of data sharing across institutions without patient information confidentiality issues.
“…e Data augmentation for ocular surface images [ 46 ] and anterior segment OCT [ 82 ]. f Segmentation for corneal sub basal nerves in in vivo confocal microscopy images [ 37 ]. Most images were generated according to publicly available datasets and the methods of each study (some cases are based on our own dataset) Fig.…”
Section: Reviewmentioning
confidence: 99%
“…Pix2pix was successfully applied to segment the retinal nerve fiber layer, Bruch’s membrane, and choroid-sclera boundary in peripapillary retinal OCT images [ 36 ]. GAN was also applied to evaluate corneal pathological conditions using in vivo confocal microscopy images [ 37 ]. In this study, the segmentation of corneal sub basal nerves was achieved using a conditional GAN to detect corneal diseases.…”
Background
Recent advances in deep learning techniques have led to improved diagnostic abilities in ophthalmology. A generative adversarial network (GAN), which consists of two competing types of deep neural networks, including a generator and a discriminator, has demonstrated remarkable performance in image synthesis and image-to-image translation. The adoption of GAN for medical imaging is increasing for image generation and translation, but it is not familiar to researchers in the field of ophthalmology. In this work, we present a literature review on the application of GAN in ophthalmology image domains to discuss important contributions and to identify potential future research directions.
Methods
We performed a survey on studies using GAN published before June 2021 only, and we introduced various applications of GAN in ophthalmology image domains. The search identified 48 peer-reviewed papers in the final review. The type of GAN used in the analysis, task, imaging domain, and the outcome were collected to verify the usefulness of the GAN.
Results
In ophthalmology image domains, GAN can perform segmentation, data augmentation, denoising, domain transfer, super-resolution, post-intervention prediction, and feature extraction. GAN techniques have established an extension of datasets and modalities in ophthalmology. GAN has several limitations, such as mode collapse, spatial deformities, unintended changes, and the generation of high-frequency noises and artifacts of checkerboard patterns.
Conclusions
The use of GAN has benefited the various tasks in ophthalmology image domains. Based on our observations, the adoption of GAN in ophthalmology is still in a very early stage of clinical validation compared with deep learning classification techniques because several problems need to be overcome for practical use. However, the proper selection of the GAN technique and statistical modeling of ocular imaging will greatly improve the performance of each image analysis. Finally, this survey would enable researchers to access the appropriate GAN technique to maximize the potential of ophthalmology datasets for deep learning research.
“…A new DL algorithm based on GANs, an emerging DL model for the processing of medical images, was first introduced for the automated segmentation of corneal subbasal nerves in IVCM images in 2021 [ 109 ]. In comparison with the U-Net-based algorithm, the GAN-based algorithm showed a similar correlation and Bland–Altman analysis results.…”
Section: Application Of Ai In Diagnosis and Treatment Of Dedmentioning
confidence: 99%
“…In comparison with the U-Net-based algorithm, the GAN-based algorithm showed a similar correlation and Bland–Altman analysis results. The GAN-based method demonstrated higher accuracy for the segmentation of corneal nerves in IVCM images, particularly in the applied images, compared to the U-Net-based method [ 109 ]. In 2022, a new DL-based algorithm that enabled the automated segmentation and evaluation of corneal nerve fibers (CNFs) and dendritic cells (DCs) separately in IVCM images based on U-Net and Mask R-CNN architectures, respectively, was produced by Setu et al [ 110 ] In this study, both the CNF model (86.1% sensitivity and 90.1% specificity) and the DC model (89.37% precision, 94.43% recall, and 91.83% F1 score) showed reliable consistency with the manual evaluation and at a substantially higher speed, suggesting that the DL model has the potential to be integrated into the monitoring tools of DED using IVCM [ 110 ].…”
Section: Application Of Ai In Diagnosis and Treatment Of Dedmentioning
Dry eye disease (DED) is one of the most common diseases worldwide that can lead to a significant impairment of quality of life. The diagnosis and treatment of the disease are often challenging because of the lack of correlation between the signs and symptoms, limited reliability of diagnostic tests, and absence of established consensus on the diagnostic criteria. The advancement of machine learning, particularly deep learning technology, has enabled the application of artificial intelligence (AI) in various anterior segment disorders, including DED. Currently, many studies have reported promising results of AI-based algorithms for the accurate diagnosis of DED and precise and reliable assessment of data obtained by imaging devices for DED. Thus, the integration of AI into clinical approaches for DED can enhance diagnostic and therapeutic performance. In this review, in addition to a brief summary of the application of AI in anterior segment diseases, we will provide an overview of studies regarding the application of AI in DED and discuss the recent advances in the integration of AI into the clinical approach for DED.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
