Medical Staff and Resident Preferences for Using Deep Learning in Eye Disease Screening: Discrete Choice Experiment

Abstract

Background Deep learning–assisted eye disease diagnosis technology is increasingly applied in eye disease screening. However, no research has suggested the prerequisites for health care service providers and residents willing to use it. Objective The aim of this paper is to reveal the preferences of health care service providers and residents for using artificial intelligence (AI) in community-based eye disease screening, particularly their preference for accuracy. Methods Discrete choice experiments for health care providers and residents were conducted in Shanghai, China. In total, 34 medical institutions with adequate AI-assisted screening experience participated. A total of 39 medical staff and 318 residents were asked to answer the questionnaire and make a trade-off among alternative screening strategies with different attributes, including missed diagnosis rate, overdiagnosis rate, screening result feedback efficiency, level of ophthalmologist involvement, organizational form, cost, and screening result feedback form. Conditional logit models with the stepwise selection method were used to estimate the preferences. Results Medical staff preferred high accuracy: The specificity of deep learning models should be more than 90% (odds ratio [OR]=0.61 for 10% overdiagnosis; P<.001), which was much higher than the Food and Drug Administration standards. However, accuracy was not the residents’ preference. Rather, they preferred to have the doctors involved in the screening process. In addition, when compared with a fully manual diagnosis, AI technology was more favored by the medical staff (OR=2.08 for semiautomated AI model and OR=2.39 for fully automated AI model; P<.001), while the residents were in disfavor of the AI technology without doctors’ supervision (OR=0.24; P<.001). Conclusions Deep learning model under doctors’ supervision is strongly recommended, and the specificity of the model should be more than 90%. In addition, digital transformation should help medical staff move away from heavy and repetitive work and spend more time on communicating with residents.

Artificial intelligence technology for myopia challenges: A review

“…Firstly, physician and patient acceptance is a challenge. Scheetz et al (2021) showed a high rate of patient satisfaction with AI technology for ophthalmic screening, but Lin et al (2022) found that residents were “algorithm aversion” and expected more physician involvement in eye screening services. Explainable artificial intelligence (XAI) is a potential solution to open the “black box” and gain the trust of patients.…”

Artificial Intelligence in Community-Based Diabetic Retinopathy Telemedicine Screening in Urban China: Cost-effectiveness and Cost-Utility Analyses With Real-world Data

“…However, relevant evidence is still inadequate; therefore, it is difficult to determine whether this improvement is an isolated case. Previously, we implemented a discrete choice experiment in Shanghai to measure individuals' preferences for AI-based screening [34]. The results suggested that the impact of the adoption of AI on individuals' preferences may be bidirectional.…”

“…The results suggested that the impact of the adoption of AI on individuals' preferences may be bidirectional. On the one hand, algorithm aversion should be noted, which means that compared to manual grading, the residents were in disfavor of the AI-assisted screening technology [34,35]. On the other hand, the immediate feedback of retinal screening results by the adoption of AI could increase the individuals' preferences and have profound effects on participants' follow-up behavior [24,34].…”

“…On the one hand, algorithm aversion should be noted, which means that compared to manual grading, the residents were in disfavor of the AI-assisted screening technology [34,35]. On the other hand, the immediate feedback of retinal screening results by the adoption of AI could increase the individuals' preferences and have profound effects on participants' follow-up behavior [24,34]. Nevertheless, there is still a lack of empirical studies on the association between the results of feedback efficiency and residents' referral compliance.…”

Deep Learning-Based Analysis of Infrared Fundus Photography for Automated Diagnosis of Diabetic Retinopathy with Cataracts

“…AI-assisted diagnostic software has been widely used in the prevention and treatment of eye diseases in Shanghai. 5…”