Challenges and barriers of using large language models (LLM) such as ChatGPT for diagnostic medicine with a focus on digital pathology – a recent scoping review

Abstract

Background The integration of large language models (LLMs) like ChatGPT in diagnostic medicine, with a focus on digital pathology, has garnered significant attention. However, understanding the challenges and barriers associated with the use of LLMs in this context is crucial for their successful implementation. Methods A scoping review was conducted to explore the challenges and barriers of using LLMs, in diagnostic medicine with a focus on digital pathology. A comprehensive search was conducted using electronic databases, including PubMed and Google Scholar, for relevant articles published within the past four years. The selected articles were critically analyzed to identify and summarize the challenges and barriers reported in the literature. Results The scoping review identified several challenges and barriers associated with the use of LLMs in diagnostic medicine. These included limitations in contextual understanding and interpretability, biases in training data, ethical considerations, impact on healthcare professionals, and regulatory concerns. Contextual understanding and interpretability challenges arise due to the lack of true understanding of medical concepts and lack of these models being explicitly trained on medical records selected by trained professionals, and the black-box nature of LLMs. Biases in training data pose a risk of perpetuating disparities and inaccuracies in diagnoses. Ethical considerations include patient privacy, data security, and responsible AI use. The integration of LLMs may impact healthcare professionals’ autonomy and decision-making abilities. Regulatory concerns surround the need for guidelines and frameworks to ensure safe and ethical implementation. Conclusion The scoping review highlights the challenges and barriers of using LLMs in diagnostic medicine with a focus on digital pathology. Understanding these challenges is essential for addressing the limitations and developing strategies to overcome barriers. It is critical for health professionals to be involved in the selection of data and fine tuning of the models. Further research, validation, and collaboration between AI developers, healthcare professionals, and regulatory bodies are necessary to ensure the responsible and effective integration of LLMs in diagnostic medicine.

Applications of Large Language Models in Pathology

“…Additional guardrails and training incorporating human feedback is envisioned to improve LLM and MLLM performance, while reducing the incidence of erroneous responses [93]. Some errors produced by LLMs may be attributable to biases and errors present in its training set, so efforts will be undertaken to improve training data quality [111]. More publicly available pathology text and text/image datasets will become available, which will open up opportunities for training more accurate and powerful models.…”

Exploring the role of Large Language Models (LLMs) in hematology: a systematic review of applications, benefits, and limitations

“…Doubts also persist about the dependability of their outputs for making clinical decisions. 6 7 As LLMs become more common in healthcare, the necessity to test their applications increases. This review evaluates the application of LLMs in the field of hematology, systematically assessing their benefits, limitations, and potential risks in medical training, education, and diagnosis.…”

Large language models in physical therapy: time to adapt and adept

“…In this way, LLM responses can be modulated and compared after being fed with accurate data and evidence-based clinical practice guidelines to meet patient needs ( 26 ). A recent scoping review by Ullah et al assessed the challenges and barriers to using LLMs in diagnostic medicine in the field of pathology ( 27 ). Many language models, such as Claude, Command, and Bloomz, have been programmed for creating accurate medical advice ( 28 ) ( Figure 2 ).…”