Purpose To quantitatively evaluate the inter-annotator variability of clinicians tracing the contours of anatomical layers of the iridocorneal angle on digital gonio photographs, thus providing a baseline for the validation of automated analysis algorithms. Methods Using a software annotation tool on a common set of 20 images, five experienced ophthalmologists highlighted the contours of five anatomical layers of interest: iris root (IR), ciliary body band (CBB), scleral spur (SS), trabecular meshwork (TM), and cornea (C). Inter-annotator variability was assessed by (1) comparing the number of times ophthalmologists delineated each layer in the dataset; (2) quantifying how the consensus area for each layer (i.e., the intersection area of observers’ delineations) varied with the consensus threshold; and (3) calculating agreement among annotators using average per-layer precision, sensitivity, and Dice score. Results The SS showed the largest difference in annotation frequency (31%) and the minimum overall agreement in terms of consensus size (∼28% of the labeled pixels). The average annotator's per-layer statistics showed consistent patterns, with lower agreement on the CBB and SS (average Dice score ranges of 0.61–0.7 and 0.73–0.78, respectively) and better agreement on the IR, TM, and C (average Dice score ranges of 0.97–0.98, 0.84–0.9, and 0.93–0.96, respectively). Conclusions There was considerable inter-annotator variation in identifying contours of some anatomical layers in digital gonio photographs. Our pilot indicates that agreement was best on IR, TM, and C but poorer for CBB and SS. Translational Relevance This study provides a comprehensive description of inter-annotator agreement on digital gonio photographs segmentation as a baseline for validating deep learning models for automated gonioscopy.
Tackling visual impairment remains an important public health issue. Due to limited resources and the increasing demand on hospital eye services (HES), delivery of quality eye care within the community is essential. Training of clinical ophthalmic specialists and allied health-care professionals in the detection and management of common eye conditions can thus help to reduce the burden of eye disease and improve prognostic outcomes. Digital imaging has become a useful tool in facilitating eye-care delivery in both the community and hospital setting. In the last decade, the advent of electronic image exchange via a centralized referral unit in Scotland has revolutionized screening for ophthalmic disease, referrals, and shared care between community and HES clinicians. A government-led initiative known as the Scottish Eyecare Integration Project introduced electronic transfer of digital images within referrals from community optometrists to HES, which greatly reduced outpatient waiting times and improved patient satisfaction. The catalogue of live clinical information and digital images that resulted from the project led to the creation of a virtual learning platform through the University of Edinburgh. Participating professionals involved in eye care have interactive discussions about common eye conditions by sharing digital images of cases and investigations on a global online platform. This has received worldwide attention and inspired the creation of other university courses, e-learning platforms in eye-health education, and shared-care schemes in the screening of eye disease. We show that digital ophthalmology plays a vital role in the integration of community and HES partnership in delivery of patient care and in facilitating eye-health education to a global audience.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.