Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

Kung, Tiffany H; Cheatham, Morgan; Medenilla, Arielle; Sillos, Czarina; Leon, Lorie De; Elepaño, Camille; Madriaga, Maria; Aggabao, Rimel; Diaz-Candido, Giezel; Maningo, James; Tseng, Victor

doi:10.1371/journal.pdig.0000198

Cited by 1,438 publications

(1,042 citation statements)

References 20 publications

Supporting

Mentioning

841

Contrasting

Unclassified

Order By: Relevance

“…This raises concerns about fair access to the same educational material despite using the same prompt. For instance, Kung et al (2023) found the accuracy of ChatGPT to be around 60%, demanding careful assessment of its output before use. Therefore, more research should be focused on ensuring fairness, accuracy, and equity among students using chatbots generally and ChatGPT particularly, which might be achieved through, for instance, having transparent and open algorithms (Bulathwela et al, 2020).…”

Section: Nothing Should Be Taken For Grantedmentioning

confidence: 99%

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

Tlili

Shehata

Adarkwah

et al. 2023

Smart Learn. Environ.

413

146

View full text Add to dashboard Cite

Artificial Intelligence (AI) technologies have been progressing constantly and being more visible in different aspects of our lives. One recent phenomenon is ChatGPT, a chatbot with a conversational artificial intelligence interface that was developed by OpenAI. As one of the most advanced artificial intelligence applications, ChatGPT has drawn much public attention across the globe. In this regard, this study examines ChatGPT in education, among early adopters, through a qualitative instrumental case study. Conducted in three stages, the first stage of the study reveals that the public discourse in social media is generally positive and there is enthusiasm regarding its use in educational settings. However, there are also voices who are approaching cautiously using ChatGPT in educational settings. The second stage of the study examines the case of ChatGPT through lenses of educational transformation, response quality, usefulness, personality and emotion, and ethics. In the third and final stage of the study, the investigation of user experiences through ten educational scenarios revealed various issues, including cheating, honesty and truthfulness of ChatGPT, privacy misleading, and manipulation. The findings of this study provide several research directions that should be considered to ensure a safe and responsible adoption of chatbots, specifically ChatGPT, in education.

show abstract

Section: Nothing Should Be Taken For Grantedmentioning

confidence: 99%

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

Tlili

Shehata

Adarkwah

et al. 2023

Smart Learn. Environ.

413

146

View full text Add to dashboard Cite

show abstract

“…Moreover, ChatGPT showed moderate accuracy to determine the imaging steps needed in breast cancer screening and evaluation of breast pain which can be a promising application in decision making in radiology [50]. There is also the prospects of personalized medicine and improved health literacy by providing easily accessible and understandable health information for the general public [52,54,59,61,72]. This utility was demonstrated by ChatGPT responses highlighting the need to consult healthcare providers among other reliable sources on specific situations [16,56].…”

Section: Discussionmentioning

confidence: 99%

The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations

Sallam¹

2023

Preprint

View full text Add to dashboard Cite

An artificial intelligence (AI)-based conversational large language model (LLM) was launched in November 2022 namely, “ChatGPT”. Despite the wide array of potential applications of LLMs in healthcare education, research and practice, several valid concerns were raised. The current systematic review aimed to investigate the possible utility of ChatGPT and to highlight its limitations in healthcare education, research and practice. Using the PRIMSA guidelines, a systematic search was conducted to retrieve English records in PubMed/MEDLINE and Google Scholar under the term “ChatGPT”. Eligibility criteria included the published research or preprints of any type that discussed ChatGPT in the context of healthcare education, research and practice. A total of 280 records were identified, and following full screening, a total of 60 records were eligible for inclusion. Benefits/applications of ChatGPT were cited in 51/60 (85.0%) records with the most common being the utility in scientific writing followed by benefits in healthcare research (efficient analysis of massive datasets, code generation and rapid concise literature reviews besides utility in drug discovery and development). Benefits in healthcare practice included cost saving, documentation, personalized medicine and improved health literacy. Concerns/possible risks of ChatGPT use were expressed in 58/60 (96.7%) records with the most common being the ethical issues including the risk of bias, plagiarism, copyright issues, transparency issues, legal issues, lack of originality, incorrect responses, limited knowledge, and inaccurate citations. Despite the promising applications of ChatGPT which can result in paradigm shifts in healthcare education, research and practice, the embrace of this application should be done with extreme caution. Specific applications of ChatGPT in health education include the promising utility in personalized learning tools and shift towards more focus on critical thinking and problem-based learning. In healthcare practice, ChatGPT can be valuable for streamlining the workflow and refining personalized medicine. Saving time for the focus on experimental design and enhancing research equity and versatility are the benefits in scientific research. Regarding authorship in scientific articles, as it currently stands, ChatGPT does not qualify to be listed as an author unless the ICJME/COPE guidelines are revised and amended. An initiative involving all stakeholders involved in healthcare education, research and practice is urgently needed to set a code of ethics and conduct on the responsible practices involving ChatGPT among other LLMs.

show abstract

“…ChatGPT will change the pedagogy in management education. For example, formative assessments (including assignment reports) to evaluate how someone is learning from the material throughout a course is not a promising idea now (Kung et al, 2023). It is time to define guidelines to use ChatGPT in education.…”

Section: Role Of Ai and Gpt In Academic Practicesmentioning

confidence: 99%

AI and GPT for Management Scholars and Practitioners: Guidelines and Implications

Rana¹

2023

FIIB Business Review

View full text Add to dashboard Cite

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

Cited by 1,438 publications

References 20 publications

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations

AI and GPT for Management Scholars and Practitioners: Guidelines and Implications

Contact Info

Product

Resources

About