Variability in Optical Coherence Tomography Angiography Interpretation in a Cohort of Retina Specialists

Abstract

BACKGROUND AND OBJECTIVE: To investigate the variability in optical coherence tomography angiography (OCTA) image interpretation in a cohort of retina specialists. PATIENTS AND METHODS: A survey consisting of a study set of images from 12 eyes examined by OCTA was created. Eight multiple-choice answers were provided as response options for each case. The survey was sent to 100 retina specialists, with instructions to complete the survey only if they had facility with the interpretation of OCTA images. Thirty-eight physicians completed the survey. Data generated were subsequently analyzed and interpreted. RESULTS: Krippendorff's alpha coefficients of agreement and their associated 95% confidence intervals (CIs) were utilized for statistical analyses. For the overall data, the estimated alpha coefficient was 0.366 (95% CI, 0.31–0.47). Although the estimated alpha coefficient is significant, the level of significance is considered low, as it is far from unity (0.366). Therefore, although statistically significant, the overall data did not demonstrate either high reliability or agreement in interpretation. Additional analyses evaluating the influence of years and location of practice, and frequency of OCTA use did not demonstrate a significant effect on reliability measures. CONCLUSIONS: Significant variability exists in the interpretation of OCTA images in this cohort of retina specialists. The overall data did not demonstrate high reliability or agreement in interpretation of images, suggesting the need for additional study of this nascent technology. [ Ophthalmic Surg Lasers Imaging Retina . 2019;50:344–353.]

Evaluation of Accuracy and Agreement of Optical Coherence Tomography Angiography Interpretation of Common Retinal Findings and Diagnoses

“…Additionally, the overall rate of agreement between the participants was also relatively low, i.e., 60.1%, and a significant part of the agreement was due to similarities in giving incorrect diagnoses or not knowing the diagnosis at all. These results are compatible with previously reported studies [6][7][8][9].…”

“…There were no significant differences in diagnosis accuracy or agreement rates between VR specialists, non-VR specialists, and residents. Also, there was no association with length of experience or self-reported familiarity with OCTA technology, corresponding to the findings of a previous study that evaluated OCTA interpretation by retina specialists [7].…”

“…Although OCTA is already relatively widespread in clinical practice, there are few studies focused on its interpretation, and a recent comprehensive one reported a high variability between retinal specialists [7]. Our study has several advantages over previously published works on this topic, as it was focused on commonly encountered retinal conditions and simulated the clinical reality of OCTA interpretation better as participants could give any response and were not limited by multiple-choice options.…”

“…Additionally, several of them had been biased as additional clinical or imaging data was available to the interpreting physicians. Considerable variability in OCTA interpretation has also been shown in a cohort of retina specialists [7]. Based on a multiple-choice questionnaire presented to retina specialists, results showed rates of correct diagnosis to vary between 0 to 76.5%, with low rates of intergrader reliability and agreement.…”

The Effect of Increasing Acquisition Speed on Optical Coherence Tomography Angiography Images

“…1,2 There are many reports on different factors influencing quality and diagnostic abilities of this noninvasive technology, including a high interobserver variability of interpretation. 15 In addition, differences in axial 16 and lateral resolution, 17 acquisition protocol, 18 field of view, or used algorithm 19 directly affect the information provided by OCTA images, and frequent adjustments have been implemented by device manufacturers to optimize the clinical performance of OCTA. Acquisition speed is also an important factor influencing the use of OCTA in daily practice.…”