Writing Multiple Choice Questions—Has the Student Become the Master?

Pham, Hannah; Court‐Kowalski, Stefan; Chan, Hong Kei; Devitt, Peter G.

doi:10.1080/10401334.2022.2050240

Cited by 2 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, repeat items in examinations from year to year, or inherent imperfections called item-writing flaws (IWFs). Although at first sight IWFs may appear trivial, they can affect the way students understand and answer questions [6, 7, 8, 9]. Producing MCQs is also time consuming.…”

Section: Introductionmentioning

confidence: 99%

Large language models for generating medical examinations: systematic review

Artsi,

Sorin,

Konen

et al. 2024

Preprint

View full text Add to dashboard Cite

PurposeWriting multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive medical knowledge, time and effort from medical educators. This systematic review focuses on the application of large language models (LLMs) in generating medical MCQs.MethodsThe authors searched for studies published up to November 2023. Search terms focused on LLMs generated MCQs for medical examinations. MEDLINE was used as a search database.ResultsOverall, eight studies published between April 2023 and October 2023 were included. Six studies used Chat-GPT 3.5, while two employed GPT 4. Five studies showed that LLMs can produce competent questions valid for medical exams. Three studies used LLMs to write medical questions but did not evaluate the validity of the questions. One study conducted a comparative analysis of different models. One other study compared LLM-generated questions with those written by humans.All studies presented faulty questions that were deemed inappropriate for medical exams. Some questions required additional modifications in order to qualify.ConclusionsLLMs can be used to write MCQs for medical examinations. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed. Until then, LLMs may serve as a supplementary tool for writing medical examinations.

show abstract

Section: Introductionmentioning

confidence: 99%

Large language models for generating medical examinations: systematic review

Artsi,

Sorin,

Konen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, repeat items in examinations from year to year, or inherent imperfections called item-writing aws (IWFs). Although at rst sight IWFs may appear trivial, they can affect the way students understand and answer questions [6,7,8,9]. Producing MCQs is also time consuming.…”

Section: Introductionmentioning

confidence: 99%

Large language models for generating medical examinations: systematic review

Artsi,

Sorin,

Konen

et al. 2024

Preprint

View full text Add to dashboard Cite

Background Writing multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive medical knowledge, time and effort from medical educators. This systematic review focuses on the application of large language models (LLMs) in generating medical MCQs. Methods The authors searched for studies published up to November 2023. Search terms focused on LLMs generated MCQs for medical examinations. MEDLINE was used as a search database. Results Overall, eight studies published between April 2023 and October 2023 were included. Six studies used Chat-GPT 3.5, while two employed GPT 4. Five studies showed that LLMs can produce competent questions valid for medical exams. Three studies used LLMs to write medical questions but did not evaluate the validity of the questions. One study conducted a comparative analysis of different models. One other study compared LLM-generated questions with those written by humans. All studies presented faulty questions that were deemed inappropriate for medical exams. Some questions required additional modifications in order to qualify. Conclusions LLMs can be used to write MCQs for medical examinations. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed. Until then, LLMs may serve as a supplementary tool for writing medical examinations.

show abstract

Writing Multiple Choice Questions—Has the Student Become the Master?

Cited by 2 publications

References 33 publications

Large language models for generating medical examinations: systematic review

Large language models for generating medical examinations: systematic review

Large language models for generating medical examinations: systematic review

Contact Info

Product

Resources

About