Reliability of Retinal Pathology Quantification in Age-Related Macular Degeneration: Implications for Clinical Trials and Machine Learning Applications

Abstract

Purpose To investigate the interreader agreement for grading of retinal alterations in age-related macular degeneration (AMD) using a reading center setting. Methods In this cross-sectional case series, spectral-domain optical coherence tomography (OCT; Topcon 3D OCT, Tokyo, Japan) scans of 112 eyes of 112 patients with neovascular AMD (56 treatment naive, 56 after three anti–vascular endothelial growth factor injections) were analyzed by four independent readers. Imaging features specific for AMD were annotated using a novel custom-built annotation platform. Dice score, Bland-Altman plots, coefficients of repeatability, coefficients of variation, and intraclass correlation coefficients were assessed. Results Loss of ellipsoid zone, pigment epithelium detachment, subretinal fluid, and drusen were the most abundant features in our cohort. Subretinal fluid, intraretinal fluid, hypertransmission, descent of the outer plexiform layer, and pigment epithelium detachment showed highest interreader agreement, while detection and measures of loss of ellipsoid zone and retinal pigment epithelium were more variable. The agreement on the size and location of the respective annotation was more consistent throughout all features. Conclusions The interreader agreement depended on the respective OCT-based feature. A selection of reliable features might provide suitable surrogate markers for disease progression and possible treatment effects focusing on different disease stages. Translational Relevance This might give opportunities for a more time- and cost-effective patient assessment and improved decision making as well as have implications for clinical trials and training machine learning algorithms.

Intersession Repeatability of Structural Biomarkers in Early and Intermediate Age-Related Macular Degeneration: A MACUSTAR Study Report

“…Among previous studies, Corvi et al 11 showed comparable intersession agreement (κ = 0.88) on assessing central drusen volume within the central 3 mm of the Early Treatment Diabetic Retinopathy Study grid. Recently, Müller et al 21 reported an intraclass correlation coefficient of 0.788 between readers for drusen detection with a minimum size of 1558.6 µm 2 , which corresponds to the minimal drusen diameter of 63 µm and is well comparable to results in this study. Compared with the previous analysis, it is important to point out that we did not only perform an independent grading of the imaging data acquired at a single time point in a single-center setting.…”

Quantitative assessment of retinal fluid in neovascular age-related macular degeneration under anti-VEGF therapy

“… 7 Discrepancies in the detection of IRF between the investigators and the reading center was also seen in the FLUID study and confirmed in an independent cohort. 22 , 23 These findings support the implementation of quantitative markers and even more preferred, an automated identification and standardized quantification of macular fluid using the entire OCT volume.…”

“…Interreader agreement is good for SRF; however, IRF might be missed, especially in scans without abundant central IRF. 23 The main limitation for manual readings, however, is the constriction to smaller data sets due to the enormous time consumption when performing precise annotation. To guarantee precise and repeatable measurements in a feasible amount of time, automatic segmentation of macular fluid has to be implemented.…”

A Deep Learning Framework for the Detection and Quantification of Reticular Pseudodrusen and Drusen on Optical Coherence Tomography

“…For example, in the Laser Intervention in Early Stages of Age-Related Macular Degeneration (LEAD) study, aimed to evaluate the safety and efficacy of subthreshold nanosecond laser in intermediate AMD, it was found that such treatment may be inappropriate in patients with RPD compared to those without. 15 This suggests that treatments for those with RPD might need to be different from those with drusen, and as such it will be important to be able to accurately and quickly identify patients with or without RPD.…”

“… 2 Given the subtlety of RPD lesions on OCT, even on the latest generation devices, and more so on early generation OCT utilized in existing large population studies, human detection and quantification remain a challenge. 15 Given the importance of being able to detect and quantify RPD and separate them from drusen in terms of both our understanding of the pathogenesis of RPD and the potential implication of their presence in current and future therapies, 16 an automated approach to classification and quantification is needed.…”