Visualizing and understanding inherent features in <scp>SD‐OCT</scp> for the progression of age‐related macular degeneration using deconvolutional neural networks

Self Cite

Age-related macular degeneration (AMD) and Stargardt disease are the leading causes of blindness for the elderly and young adults respectively. Geographic atrophy (GA) of AMD and Stargardt atrophy are their end-stage outcomes. Efficient methods for segmentation and quantification of these atrophic lesions are critical for clinical research. In this study, we developed a deep convolutional neural network (CNN) with a trainable self-attended mechanism for accurate GA and Stargardt atrophy segmentation. Compared with traditional post-hoc attention mechanisms which can only visualize CNN features, our self-attended mechanism is embedded in a fully convolutional network and directly involved in training the CNN to actively attend key features for enhanced algorithm performance. We applied the self-attended CNN on the segmentation of AMD and Stargardt atrophic lesions on fundus autofluorescence (FAF) images. Compared with a preexisting regular fully convolutional network (the U-Net), our self-attended CNN achieved 10.6% higher Dice coefficient and 17% higher IoU (intersection over union) for AMD GA segmentation, and a 22% higher Dice coefficient and a 32% higher IoU for Stargardt atrophy segmentation. With longitudinal image data having over a longer time, the developed self-attended mechanism can also be applied on the visual discovery of early AMD and Stargardt features.

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Automated segmentation and feature discovery of age-related macular degeneration and Stargardt disease via self-attended neural networks

Wang

Sadda

Lee

et al. 2022

Self Cite

“…Visualization methods aim to produce “visual explanations” for decisions of CNN-based models to make them more transparent and explainable 38 . In order to build trust in intelligent systems a number of visualization methods have been proposed recently so that the input stimuli that excite CNNs could be made visually transparent 39 .…”

Section: Literature Reviewmentioning

confidence: 99%

Understanding inherent image features in CNN-based assessment of diabetic retinopathy

Reguant

Brunak

Saha

2021

Self Cite

Diabetic retinopathy (DR) is a leading cause of blindness and affects millions of people throughout the world. Early detection and timely checkups are key to reduce the risk of blindness. Automated grading of DR is a cost-effective way to ensure early detection and timely checkups. Deep learning or more specifically convolutional neural network (CNN)—based methods produce state-of-the-art performance in DR detection. Whilst CNN based methods have been proposed, no comparisons have been done between the extracted image features and their clinical relevance. Here we first adopt a CNN visualization strategy to discover the inherent image features involved in the CNN’s decision-making process. Then, we critically analyze those features with respect to commonly known pathologies namely microaneurysms, hemorrhages and exudates, and other ocular components. We also critically analyze different CNNs by considering what image features they pick up during learning to predict and justify their clinical relevance. The experiments are executed on publicly available fundus datasets (EyePACS and DIARETDB1) achieving an accuracy of 89 ~ 95% with AUC, sensitivity and specificity of respectively 95 ~ 98%, 74 ~ 86%, and 93 ~ 97%, for disease level grading of DR. Whilst different CNNs produce consistent classification results, the rate of picked-up image features disagreement between models could be as high as 70%.

“…In particular, previous work in our group has demonstrated that such deep learning methods are able to detect GA in late stage AMD 8 and identify known baseline AMD biomarkers 6 . Further, our group has demonstrated that the combination of deconvolutional networks with a U-Net architecture 8 , 9 can reveal features most salient for anticipating the progression of AMD 7 . A modified U-Net model was trained to predict GA progression from en-face projections of OCT volumes with ground truth labels provided by annotated FAF images from a follow-up visit.…”

Section: Introductionmentioning

confidence: 98%

“…Recently, there has been work towards applying deep learning methods to identify and visualize AMD baseline biomarkers 5 – 7 and AMD late stage atrophy 8 in SD-OCT and FAF images that can diagnose AMD. Deep learning—an objective artificial intelligence approach, has an advantage over traditional machine learning methods in that it learns directly from data so there is no need to manually designate features.…”

Section: Introductionmentioning

confidence: 99%

Reverse engineering for reconstructing baseline features of dry age-related macular degeneration in optical coherence tomography

Wang

Vejalla

et al. 2022

Self Cite

Age-related macular degeneration (AMD) is the most widespread cause of blindness and the identification of baseline AMD features or biomarkers is critical for early intervention. Optical coherence tomography (OCT) imaging produces a 3D volume consisting of cross sections of retinal tissue while fundus fluorescence (FAF) imaging produces a 2D mapping of retina. FAF has been a good standard for assessing dry AMD late-stage geographic atrophy (GA) while OCT has been used for assessing early AMD biomarkers beyond as well. However, previous approaches in large extent defined AMD features subjectively based on clinicians’ observation. Deep learning—an objective artificial intelligence approach, may enable to discover ’true’ salient AMD features. We develop a novel reverse engineering approach which bases on the backbone of a fully convolutional neural network to objectively identify and visualize AMD early biomarkers in OCT from baseline exams before significant atrophy occurs. Utilizing manually annotated GA regions on FAF from a follow-up visit as ground truth, we segment GA regions and reconstruct early AMD features in baseline OCT volumes. In this preliminary exploration, compared with ground truth, we achieve baseline GA segmentation accuracy of 0.95 and overlapping ratio of 0.65. The reconstructions consistently highlight that large druse and druse clusters with or without mixed hyper-reflective focus lesion on baseline OCT cause the conversion of GA after 12 months. However, hyper-reflective focus lesions and subretinal drusenoid deposit lesions alone are not seen such conversion after 12 months. Further research with larger dataset would be needed to verify these findings.