The Clinical Influence after Implementation of Convolutional Neural Network-Based Software for Diabetic Retinopathy Detection in the Primary Care Setting

Abstract

Deep learning-based software is developed to assist physicians in terms of diagnosis; however, its clinical application is still under investigation. We integrated deep-learning-based software for diabetic retinopathy (DR) grading into the clinical workflow of an endocrinology department where endocrinologists grade for retinal images and evaluated the influence of its implementation. A total of 1432 images from 716 patients and 1400 images from 700 patients were collected before and after implementation, respectively. Using the grading by ophthalmologists as the reference standard, the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) to detect referable DR (RDR) were 0.91 (0.87–0.96), 0.90 (0.87–0.92), and 0.90 (0.87–0.93) at the image level; and 0.91 (0.81–0.97), 0.84 (0.80–0.87), and 0.87 (0.83–0.91) at the patient level. The monthly RDR rate dropped from 55.1% to 43.0% after implementation. The monthly percentage of finishing grading within the allotted time increased from 66.8% to 77.6%. There was a wide range of agreement values between the software and endocrinologists after implementation (kappa values of 0.17–0.65). In conclusion, we observed the clinical influence of deep-learning-based software on graders without the retinal subspecialty. However, the validation using images from local datasets is recommended before clinical implementation.

Transparency in Artificial Intelligence Reporting in Ophthalmology-A Scoping Review

“…Training level characteristics are shown in Table 6 . We were able to find training information for 27 of the total 37 studies, 14 , 17 , 19 , 23 , 24 , 25 , 26 , 36 , 56 , 43 , 51 , 39 , 53 , 45 , 54 , 52 , 57 , 40 , 50 , 55 , 49 , 42 , 58 , 48 , 41 , 44 , 59 22 of which were unique AI models. Training information could not be found for the remaining 5 unique models.…”

Toward better diabetes care: Exploration and implementation

“…Furthermore, we applied an artificial intelligence‐derived software (Convolutional Neural Network), which was approved by the Taiwan Food and Drug Administration 27 to assist physicians in grading pictures of eye fundi in individuals with diabetes We attempted to determine whether the implementation of this software could help endocrinologists to finish reports early or reduce unnecessary referrals to ophthalmologists. After incorporating this artificial intelligence software into our workflow, we found that the monthly percentage of finishing grading reports of fundi pictures from patients with diabetes within the allotted time increased from 66.8% to 77.6% in diabetes clinics, whereas the monthly referral rate to ophthalmology dropped from 55.1% to 43.0% after implementation 28 .…”