ChatGPT: First glance from a perspective of clinical pharmacology

Krumborg, Julie Rudbech; Mikkelsen, Nicolaj; Damkier, Per; Ennis, Zandra Nymand; Henriksen, Daniel Pilsgaard; Lillevang‐Johansen, Mads; Pedersen, Sidsel Arnspang; Bergmann, Troels K.

doi:10.1111/bcpt.13879

Cited by 6 publications

(3 citation statements)

References 10 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ChatGPT is now employed for searching potential drug targets 13 . Additionally, it may be useful as an assistant data analysist in pharmacogenomics 14 . Another LLM developed by DeepMind, AlphaFold, excels in predicting the accurate 3‐dimensional structure of proteins from amino acid sequences 15 .…”

Section: Introductionmentioning

confidence: 99%

Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists

Huang,

Estau,

Liu

et al. 2023

Brit J Clinical Pharma

View full text Add to dashboard Cite

AimTo evaluate the performance of chat generative pre‐trained transformer (ChatGPT) in key domains of clinical pharmacy practice, including prescription review, patient medication education, adverse drug reaction (ADR) recognition, ADR causality assessment, and drug counseling.MethodsQuestions and clinical pharmacist’ answers were collected from real clinical cases and clinical pharmacist competency assessment. ChatGPT’s responses were generated by inputting the same question into the "New Chat" box of “ChatGPT Mar 23 Version”. Five licensed clinical pharmacists independently rated these answers on a scale of 0 (completely incorrect) to 10 (completely correct). The mean scores of ChatGPT and clinical pharmacists were compared using a paired two‐tailed Student's t‐test. The text content of the answers was also descriptively summarized together.ResultsThe quantitative results indicated that ChatGPT was excellent in drug counseling (ChatGPT: 8.77 vs. clinical pharmacist: 9.50, p=0.0791) and weak in prescription review (5.23 vs. 9.90, p=0.0089), patient medication education (6.20 vs. 9.07, p=0.0032), ADR recognition (5.07 vs. 9.70, p=0.0483), and ADR causality assessment (4.03 vs. 9.73, p=0.023). The capabilities and limitations of ChatGPT in clinical pharmacy practice were summarized based on the completeness and accuracy of the answers. ChatGPT revealed robust retrieval, information integration, and dialogue capabilities. It lacked medicine‐specific datasets as well as the ability for handling advanced reasoning and complex instructions.ConclusionsWhile ChatGPT holds promise in clinical pharmacy practice as a supplementary tool, the ability of ChatGPT to handle complex problems needs further improvement and refinement.

show abstract

Section: Introductionmentioning

confidence: 99%

Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists

Huang,

Estau,

Liu

et al. 2023

Brit J Clinical Pharma

View full text Add to dashboard Cite

show abstract

“…From January 1st, 2023 until July 10th, 2023, there have been over 768 publications indexed in PubMed, that contain the string “ChatGPT” and that delve into various aspects of this Generative Pre-trained Transformer (GPT) chatbot. Some of the topics of such publications include the potential impact of ChatGPT on clinical and translational medicine 3 , clinical pharmacology 4 , scientific communication 5 and medical writing 6 , medical evidence summarization 7 , patient question-answering 8 , public health 9 , ethical and legal 10 , health policy making 11 , plastic and colorectal surgery 12 , 13 , doctor-patient communication 9 , or drug-drug interaction and explainability 14 .…”

Section: Introductionmentioning

confidence: 99%

Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training

Madrid-García,

Rosales-Rosado,

Freites-Nuñez

et al. 2023

Sci Rep

View full text Add to dashboard Cite

The emergence of large language models (LLM) with remarkable performance such as ChatGPT and GPT-4, has led to an unprecedented uptake in the population. One of their most promising and studied applications concerns education due to their ability to understand and generate human-like text, creating a multitude of opportunities for enhancing educational practices and outcomes. The objective of this study is twofold: to assess the accuracy of ChatGPT/GPT-4 in answering rheumatology questions from the access exam to specialized medical training in Spain (MIR), and to evaluate the medical reasoning followed by these LLM to answer those questions. A dataset, RheumaMIR, of 145 rheumatology-related questions, extracted from the exams held between 2010 and 2023, was created for that purpose, used as a prompt for the LLM, and was publicly distributed. Six rheumatologists with clinical and teaching experience evaluated the clinical reasoning of the chatbots using a 5-point Likert scale and their degree of agreement was analyzed. The association between variables that could influence the models’ accuracy (i.e., year of the exam question, disease addressed, type of question and genre) was studied. ChatGPT demonstrated a high level of performance in both accuracy, 66.43%, and clinical reasoning, median (Q1–Q3), 4.5 (2.33–4.67). However, GPT-4 showed better performance with an accuracy score of 93.71% and a median clinical reasoning value of 4.67 (4.5–4.83). These findings suggest that LLM may serve as valuable tools in rheumatology education, aiding in exam preparation and supplementing traditional teaching methods.

show abstract

“…From January 1 st , 2023 until July 10 th , 2023, there have been over 768 publications indexed in PubMed, that contain the string ChatGPT and that delve into various aspects of this Generative Pre-trained Transformer (GPT) chatbot. Some of the topics of such publications include the potential impact of ChatGPT on clinical and translational medicine [3], clinical pharmacology [4], scientific communication [5] and medical writing [6], medical evidence summarization [7], patient question-answering [8], public health [9], ethical and legal [10], health policy making [11], plastic and colorectal surgery [12,13], doctorpatient communication [9], or drug-drug interaction and explainability [14].…”

Section: Introductionmentioning

confidence: 99%

Harnessing ChatGPT and GPT-4 for Evaluating the Rheumatology Questions of the Spanish Access Exam to Specialized Medical Training

Madrid

Rosales-Rosado

Freites

et al. 2023

Preprint

View full text Add to dashboard Cite

The emergence of Large Language Models (LLM) with remarkable performance such as ChatGPT and GPT-4, has led to an unprecedented uptake in the population. One of their most promising and studied applications concerns education due to their ability to understand and generate human-like text, creating a multitude of opportunities for enhancing educational practices and outcomes. The objective of this study is two-fold: to assess the accuracy of ChatGPT/GPT-4 in answering rheumatology questions from the access exam to specialized medical training in Spain (MIR), and to evaluate the medical reasoning followed by these LLM to answer those questions. A dataset, RheumaMIR, of 145 rheumatology-related questions, extracted from the exams held between 2010 and 2023, was created for that purpose, used as a prompt for the LLM, and was publicly distributed. Six rheumatologists with clinical and teaching experience evaluated the clinical reasoning of the chatbots using a 5-point Likert scale and their degree of agreement was analyzed. The association between variables that could influence the models' accuracy (i.e., year of the exam question, disease addressed, type of question and genre) was studied. ChatGPT demonstrated a high level of performance in both accuracy, 66.43%, and clinical reasoning, median (Q1-Q3), 4.5 (2.33-4.67). However, GPT-4 showed better performance with an accuracy score of 93.71% and a median clinical reasoning value of 4.67 (4.5-4.83). These findings suggest that LLM may serve as valuable tools in rheumatology education, aiding in exam preparation and supplementing traditional teaching methods.

show abstract

ChatGPT: First glance from a perspective of clinical pharmacology

Abstract: | SO, WHAT EXACTLY IS ChatGPT?ChatGPT is short for Chat Generative Pre-trained Transformer. It is an AI "chatbot"-software program, created

Cited by 6 publications

References 10 publications

Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists

Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists

Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training

Harnessing ChatGPT and GPT-4 for Evaluating the Rheumatology Questions of the Spanish Access Exam to Specialized Medical Training

Contact Info

Product

Resources

About