AI in health: keeping the human in the loop

Bakken, Suzanne

doi:10.1093/jamia/ocad091

Cited by 8 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, close monitoring is essential to avoid mistakes caused by inaccuracy or fact fabrication [ 32 ]. Clinical applications would also benefit from an uncertainty indicator reducing the risk of erroneous decisions [ 7 ].…”

Section: Discussionmentioning

confidence: 99%

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Thirunavukarasu,

Mahmood,

Malem

et al. 2024

PLOS Digit Health

View full text Add to dashboard Cite

Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64–90%), ophthalmology trainees (median 59%, range 57–63%), and unspecialised junior doctors (median 43%, range 41–44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p>0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p<0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted.

show abstract

Section: Discussionmentioning

confidence: 99%

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Thirunavukarasu,

Mahmood,

Malem

et al. 2024

PLOS Digit Health

View full text Add to dashboard Cite

show abstract

“…Such processes would respect patient preferences and clinical expert knowledge and could enhance the safety and acceptability of predictive models. 47 , 48 Still, while incorporating human input could have benefits, it could also introduce other implicit (or explicit) biases into addressing HRSN. Additionally, incorporating clinical expertise into the process increases the workflow redesign and integration burden.…”

Section: Discussionmentioning

confidence: 99%

Acceptance of Automated Social Risk Scoring in the Emergency Department: Clinician, Staff, and Patient Perspectives

Mazurenko,

Hirsh,

Harle

et al. 2024

Western Journal of Emergency Medicine

View full text Add to dashboard Cite

Introduction: Healthcare organizations are under increasing pressure from policymakers, payers, and advocates to screen for and address patients’ health-related social needs (HRSN). The emergency department (ED) presents several challenges to HRSN screening, and patients are frequently not screened for HRSNs. Predictive modeling using machine learning and artiﬁcial intelligence, approaches may address some pragmatic HRSN screening challenges in the ED. Because predictive modeling represents a substantial change from current approaches, in this study we explored the acceptability of HRSN predictive modeling in the ED. Methods: Emergency clinicians, ED staff, and patient perspectives on the acceptability and usage of predictive modeling for HRSNs in the ED were obtained through in-depth semi-structured interviews (eight per group, total 24). All participants practiced at or had received care from an urban, Midwest, safety-net hospital system. We analyzed interview transcripts using a modiﬁed thematic analysis approach with consensus coding. Results: Emergency clinicians, ED staff, and patients agreed that HRSN predictive modeling must lead to actionable responses and positive patient outcomes. Opinions about using predictive modeling results to initiate automatic referrals to HRSN services were mixed. Emergency clinicians and staff wanted transparency on data inputs and usage, demanded high performance, and expressed concern for unforeseen consequences. While accepting, patients were concerned that prediction models can miss individuals who required services and might perpetuate biases. Conclusion: Emergency clinicians, ED staff, and patients expressed mostly positive views about using predictive modeling for HRSNs. Yet, clinicians, staff, and patients listed several contingent factors impacting the acceptance and implementation of HRSN prediction models in the ED.

show abstract

“…Such an explainable computer-aided tool will be highly beneficial for medical practitioners in validating their decisions and expanding the knowledge base with the help of AI. Such an AI and human-in-the-loop model adaptively amalgamates human knowledge as well as AI tools thus bridging the existing semantic gap between man and machine and can instill new interests in the multi-disciplinary research community of AI and medicine (Bakken, 2023 ).…”

Section: Discussionmentioning

confidence: 99%

“…As pointed out in the previous section, it is found that there exists a semantic gap between the practical use of AI in medicine and clinical decisions. We foresee that incorporating explainable computer-aided tools in medicine can complement the medical practitioners in validating their decisions in a better interpretable way by facilitating “AI-&-human-in-the-loop” thus fusing the best of both worlds (Bakken, 2023 ). More research in this direction is expected in the future.…”

Section: Discussionmentioning

confidence: 99%

Model-agnostic explainable artificial intelligence tools for severity prediction and symptom analysis on Indian COVID-19 data

Nambiar,

2023

Front. Artif. Intell.

View full text Add to dashboard Cite

IntroductionThe COVID-19 pandemic had a global impact and created an unprecedented emergency in healthcare and other related frontline sectors. Various Artificial-Intelligence-based models were developed to effectively manage medical resources and identify patients at high risk. However, many of these AI models were limited in their practical high-risk applicability due to their “black-box” nature, i.e., lack of interpretability of the model. To tackle this problem, Explainable Artificial Intelligence (XAI) was introduced, aiming to explore the “black box” behavior of machine learning models and offer definitive and interpretable evidence. XAI provides interpretable analysis in a human-compliant way, thus boosting our confidence in the successful implementation of AI systems in the wild.MethodsIn this regard, this study explores the use of model-agnostic XAI models, such as SHapley Additive exPlanations values (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME), for COVID-19 symptom analysis in Indian patients toward a COVID severity prediction task. Various machine learning models such as Decision Tree Classifier, XGBoost Classifier, and Neural Network Classifier are leveraged to develop Machine Learning models.Results and discussionThe proposed XAI tools are found to augment the high performance of AI systems with human interpretable evidence and reasoning, as shown through the interpretation of various explainability plots. Our comparative analysis illustrates the significance of XAI tools and their impact within a healthcare context. The study suggests that SHAP and LIME analysis are promising methods for incorporating explainability in model development and can lead to better and more trustworthy ML models in the future.

show abstract

AI in health: keeping the human in the loop

Cited by 8 publications

References 7 publications

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Acceptance of Automated Social Risk Scoring in the Emergency Department: Clinician, Staff, and Patient Perspectives

Model-agnostic explainable artificial intelligence tools for severity prediction and symptom analysis on Indian COVID-19 data

Contact Info

Product

Resources

About