Emergent Ophthalmic Disease Knowledge among Non-Ophthalmologist Healthcare Professionals in the Western Region of Saudi Arabia: Cross-Sectional Study

Alsaedi, Mohammed G.; Alhujaili, Haneen Omar; Fairaq, Ghadeer Saleh; Alwdaan, Sohad Ali; Alwadan, Rwaa Ali

doi:10.2174/18743641-v16-e2203160

Cited by 3 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further improvement in performance may be obtained by specific fine-tuning of models with high quality ophthalmological text data, requiring curation and deidentification [ 29 ]. GPT-4 may prove especially useful where access to ophthalmologists is limited: provision of advice, diagnosis, and management suggestions by a model with FRCOphth Part 2-level knowledge and reasoning ability is likely to be superior to non-specialist doctors and allied healthcare professionals working without support, as their exposure to and knowledge of eye care is limited [ 27 , 30 , 31 ].…”

Section: Discussionmentioning

confidence: 99%

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Thirunavukarasu,

Mahmood,

Malem

et al. 2024

PLOS Digit Health

View full text Add to dashboard Cite

Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64–90%), ophthalmology trainees (median 59%, range 57–63%), and unspecialised junior doctors (median 43%, range 41–44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p>0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p<0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted.

show abstract

Section: Discussionmentioning

confidence: 99%

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Thirunavukarasu,

Mahmood,

Malem

et al. 2024

PLOS Digit Health

View full text Add to dashboard Cite

show abstract

Knowledge, attitude, and practice of non-ophthalmic medical staff toward myopia-related fundus lesions

Wang,

You,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

This study assessed the knowledge, attitude, and practice (KAP) of non-ophthalmic medical staff towards myopia-related fundus lesions. This multicenter, cross-sectional study enrolled non-ophthalmic medical staff of Suining City between January and May 2023 using a self-designed questionnaire. A total of 505 (93.19%) valid questionnaires were included. Their mean KAP scores were 8.10 ± 2.32 (range: 0–12), 20.27 ± 2.68 (range: 0–24), and 17.77 ± 5.04 (range: 0–28), respectively. Structural equation modeling indicated that knowledge has a positive effect on attitude (β = 0.307, P < 0.001), and attitude has a positive effect on practice (β = 0.604, P < 0.001). Moreover, a higher degree of myopia exhibited a positive effect on knowledge (β = 0.510, P < 0.001). Nurses and other medical staff showed a negative effect on knowledge (β = − 0.706, P < 0.001) compared to doctors. Working in secondary and tertiary public hospitals, as well as private hospitals, demonstrated a negative effect on practice (β = − 1.963, P < 0.001) compared to those working in primary hospitals. Non-ophthalmic medical staff exhibited moderate knowledge, positive attitudes, and moderate practices toward myopia-related fundus lesions. The degree of myopia, doctors vs. other medical staff, and the hospital level influence the KAP of non-ophthalmic medical staff.

show abstract

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Thirunavukarasu,

Mahmood,

Malem

et al. 2023

Preprint

View full text Add to dashboard Cite

ObjectiveTo evaluate the clinical potential of large language models (LLMs) in ophthalmology using a more robust benchmark than raw examination scores.Materials and methodsGPT-3.5 and GPT-4 were trialled on 347 questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions.ResultsThe performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64-90%), ophthalmology trainees (median 59%, range 57-63%), and unspecialised junior doctors (median 43%, range 41-44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p>0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p<0.05).DiscussionIn view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted.ConclusionLLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. Further research is required to develop and validate clinical applications to improve eye health outcomes.

show abstract

Emergent Ophthalmic Disease Knowledge among Non-Ophthalmologist Healthcare Professionals in the Western Region of Saudi Arabia: Cross-Sectional Study

Cited by 3 publications

References 20 publications

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Knowledge, attitude, and practice of non-ophthalmic medical staff toward myopia-related fundus lesions

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Contact Info

Product

Resources

About