Comparing humans and deep learning performance for grading AMD: A study in using universal deep features and transfer learning for automated AMD analysis

Abstract

Background When left untreated, age-related macular degeneration (AMD) is the leading cause of vision loss in people over fifty in the US. Currently it is estimated that about eight million US individuals have the intermediate stage of AMD that is often asymptomatic with regard to visual deficit. These individuals are at high risk for progressing to the advanced stage where the often treatable choroidal neovascular form of AMD can occur. Careful monitoring to detect the onset and prompt treatment of the neovascular form as well as dietary supplementation can reduce the risk of vision loss from AMD, therefore, preferred practice patterns recommend identifying individuals with the intermediate stage in a timely manner. Methods Past automated retinal image analysis (ARIA) methods applied on fundus imagery have relied on engineered and hand-designed visual features. We instead detail the novel application of a machine learning approach using deep learning for the problem of ARIA and AMD analysis. We use transfer learning and universal features derived from deep convolutional neural networks (DCNN). We address clinically relevant 4-class, 3-class, and 2-class AMD severity classification problems. Results Using 5664 color fundus images from the NIH AREDS dataset and DCNN universal features, we obtain values for accuracy for the (4-,3-,2-) class classification problem of (79.4%, 81.5%, 93.4%) for machine vs. (75.8%, 85.0%, 95.2%) for physician grading. Discussion This study demonstrates the efficacy of machine grading based on deep universal features/transfer learning when applied to ARIA and is a promising step in providing a pre-screener to identify individuals with intermediate AMD and also as a tool that can facilitate identifying such individuals for clinical studies aimed at developing improved therapies. It also demonstrates comparable performance between computer and physician grading.

Current state and future prospects of artificial intelligence in ophthalmology: a review

“…Schmidt-Erfurth et al aimed to predict best-corrected visual acuity at 1 year from baseline. 51 These results are consistent with another study by Burlina et al, which aimed to solve a 2-class AMD classification problem for the purposes of triage and referral. 49 Aslam et al managed to achieve a predictive MLC that had a root mean squared error of 8.21 letters (95% confidence interval of prediction within 16.4 letters).…”

Development and validation of a deep‐learning algorithm for the detection of neovascular age‐related macular degeneration from colour fundus photographs

“…Greater numbers of classification groups led to lower kappa scores in the study by Burlina et al (2018) (0.77 for four‐step approach and 0.74 in the nine‐step approach) and an overall accuracy of 63% in the study by Grassman et al (2018) . Similarly, this was shown by Burlina et al (2017) who obtained accuracy values of 79.4%, 81.5% and 93.4% for 4‐class, 3‐class and 2‐class classifications, respectively. Rather than using a multi‐step approach, Peng et al (2018) detected individual AMD risk factors including drusen, pigmentary changes and late AMD.…”

“…Over the past decade, automated techniques for the assessment of AMD, via feature extraction from small retinal image datasets (<1000), have been reported with variable accuracy. More recent reports provide novel data on the accuracy of deep‐learning systems for the detection of AMD . The majority of these studies have utilized the Age‐Related Eye Disease Study (AREDS) participants to develop and test the accuracy of their DLAs .…”

“…More recent reports provide novel data on the accuracy of deep‐learning systems for the detection of AMD . The majority of these studies have utilized the Age‐Related Eye Disease Study (AREDS) participants to develop and test the accuracy of their DLAs . The AREDS dataset is useful for training and internally validating DLAs as the images are of high‐quality (pupillary dilation ≥5 mm), are collected within research settings and have already been gold standard graded by experts.…”

“…Direct comparison between recent reports relating to AMD detection using DLAs is made difficult due to differing classification criteria. Burlina et al (2018), Grassman et al (2018) and Burlina et al (2017) utilized multi‐step approaches using classifications developed for AREDS. For example, Grassman et al defined 13 classes, one indicating little or no AMD, grades 2 to 9 representing changes associated with early or intermediate AMD and grades 10 to 12 covering late‐stage AMD such as geographic atrophy and neovascular AMD.…”