Interexpert Agreement in the Identification of Macular Location in Infants at Risk for Retinopathy of Prematurity

Abstract

To characterize variability in the identification of the macular center among retinopathy of prematurity (ROP) experts. Methods: A printed set of 25 wide-angle retinal images was compiled from infants at risk for ROP using a commercially available camera. Ten recognized ROP experts were asked to mark the macular center on each image. For each image, we measured the distance from the optic disc center to the marked macular center. Distances were standardized by normalizing the horizontal optic disc diameter in each image to 0.93 mm. In images with visible peripheral disease, interexpert agreement on the presence of zone I disease was also determined. Results: For the image with the least variability among experts, mean (SD) distance from the optic disc to the macu-lar center was 3.69 (0.21) mm (range, 3.13-3.81 mm). For the image with the greatest variability among experts, distance from the optic disc to the macular center was 4.32 (1.19) mm (range, 3.21-7.19 mm). In 7 of 21 images (33%) with visible peripheral disease, there would have been disagreement among experts in the diagnosis of zone I disease based on identification of the macular center. Among the 10 experts, in 17 of 25 images (68%), 1 expert identified the distance between the optic disc and macular center to be greater than 1 SD from the mean. Conclusion: Significant variability exists among experts in identification of the macular center from wideangle images, which raises concerns about the reliability of zone I ROP diagnosis.

Plus Disease in Retinopathy of Prematurity

“…Similarly, other infants may be over-treated and subjected to unnecessary treatment with associated morbidity of laser and/or anti-vascular endothelial growth factor treatment. This is consistent with previously-published studies that have shown diagnostic inconsistency among experts, 6,7,15,20–23 and reinforces the importance of developing methods to improve the accuracy and consistency of plus disease diagnosis. We feel that computer-based image analysis methods 14,24–26 and tele-education 27,28 are two promising approaches to address this issue.…”

Plus Disease in Retinopathy of Prematurity

“…Similarly, other infants may be over-treated and subjected to unnecessary treatment with associated morbidity of laser and/or anti-vascular endothelial growth factor treatment. This is consistent with previously-published studies that have shown diagnostic inconsistency among experts, 6,7,15,20–23 and reinforces the importance of developing methods to improve the accuracy and consistency of plus disease diagnosis. We feel that computer-based image analysis methods 14,24–26 and tele-education 27,28 are two promising approaches to address this issue.…”

Intra and interobserver agreement in the classification of fundus autofluorescence patterns in geographic atrophy secondary to age-related macular degeneration

“…The level of diagnostic variability amongst observers found in our study is similar to that reported in other ophthalmologic disorders (for example, in evaluation of choroidal neovascularization in AMD using fluorescein angiography [14], in the evaluation of visual fields in glaucoma [15] or in the location of the macula in retinopathy of prematurity [16]), and even in other medical specialties (oncology [17] or cardiology [18], to name but a few). Variability seems unavoidable whenever a subjective judgment is required.…”

Inter-expert and Intra-expert Agreement on the Diagnosis and Treatment of Retinopathy of Prematurity

“…Although our results on inter-expert agreement on ROP disease to a certain extent confirm those reported in earlier studies 11,12,15 , we analyzed agreement on more diagnostic staging criteria of ROP disease than former studies under broader and more practice-based testing conditions: Past studies have mainly focused on central vascular changes based on single cropped images and discounted other criteria of ROP assessment like ROP stage. Inter-expert agreement on plus disease among seven ROP experts had a kappa range from 0.21 to 0.40 11 and was also low (7.38%) in a study using quadrant scores which did not report kappa values.…”

Plus Disease in Retinopathy of Prematurity

“…Similarly, other infants may be over-treated and subjected to unnecessary treatment with associated morbidity of laser and/or anti-vascular endothelial growth factor treatment. This is consistent with previously-published studies that have shown diagnostic inconsistency among experts, 6,7,15,20–23 and reinforces the importance of developing methods to improve the accuracy and consistency of plus disease diagnosis. We feel that computer-based image analysis methods 14,24–26 and tele-education 27,28 are two promising approaches to address this issue.…”

Intra and interobserver agreement in the classification of fundus autofluorescence patterns in geographic atrophy secondary to age-related macular degeneration

“…The level of diagnostic variability amongst observers found in our study is similar to that reported in other ophthalmologic disorders (for example, in evaluation of choroidal neovascularization in AMD using fluorescein angiography [14], in the evaluation of visual fields in glaucoma [15] or in the location of the macula in retinopathy of prematurity [16]), and even in other medical specialties (oncology [17] or cardiology [18], to name but a few). Variability seems unavoidable whenever a subjective judgment is required.…”

Inter-expert and Intra-expert Agreement on the Diagnosis and Treatment of Retinopathy of Prematurity

“…Although our results on inter-expert agreement on ROP disease to a certain extent confirm those reported in earlier studies 11,12,15 , we analyzed agreement on more diagnostic staging criteria of ROP disease than former studies under broader and more practice-based testing conditions: Past studies have mainly focused on central vascular changes based on single cropped images and discounted other criteria of ROP assessment like ROP stage. Inter-expert agreement on plus disease among seven ROP experts had a kappa range from 0.21 to 0.40 11 and was also low (7.38%) in a study using quadrant scores which did not report kappa values.…”

Plus Disease in Retinopathy of Prematurity

“…Similarly, other infants may be over-treated and subjected to unnecessary treatment with associated morbidity of laser and/or anti-vascular endothelial growth factor treatment. This is consistent with previously-published studies that have shown diagnostic inconsistency among experts, 6,7,15,20–23 and reinforces the importance of developing methods to improve the accuracy and consistency of plus disease diagnosis. We feel that computer-based image analysis methods 14,24–26 and tele-education 27,28 are two promising approaches to address this issue.…”

Intra and interobserver agreement in the classification of fundus autofluorescence patterns in geographic atrophy secondary to age-related macular degeneration

“…The level of diagnostic variability amongst observers found in our study is similar to that reported in other ophthalmologic disorders (for example, in evaluation of choroidal neovascularization in AMD using fluorescein angiography [14], in the evaluation of visual fields in glaucoma [15] or in the location of the macula in retinopathy of prematurity [16]), and even in other medical specialties (oncology [17] or cardiology [18], to name but a few). Variability seems unavoidable whenever a subjective judgment is required.…”

Inter-expert and Intra-expert Agreement on the Diagnosis and Treatment of Retinopathy of Prematurity

“…Although our results on inter-expert agreement on ROP disease to a certain extent confirm those reported in earlier studies 11,12,15 , we analyzed agreement on more diagnostic staging criteria of ROP disease than former studies under broader and more practice-based testing conditions: Past studies have mainly focused on central vascular changes based on single cropped images and discounted other criteria of ROP assessment like ROP stage. Inter-expert agreement on plus disease among seven ROP experts had a kappa range from 0.21 to 0.40 11 and was also low (7.38%) in a study using quadrant scores which did not report kappa values.…”