Ethical use of Artificial Intelligence in Health Professions Education: AMEE Guide No. 158

Masters, Ken

doi:10.1080/0142159x.2023.2186203

Cited by 94 publications

(75 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[25][26][27] ChatGPT may become an instrumental teaching tool to help students master the undergraduate medical curriculum, although the degree to which it can and should be incorporated into medical education has understandably become a controversial subject. Our group supports the thoughtful and ethical incorporation of artificial intelligence in medical education 28 and are actively exploring ways that these tools might be used to support underperforming students. Moreover, calls to incorporate knowledge of artificial intelligence in medical education, including appropriate use, continue to build 29,30 and there is now gradual incorporation into Canadian medical training.…”

Section: Morjaria Et Almentioning

confidence: 99%

Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program

Morjaria,

Burns,

Bracken

et al. 2023

Journal of Medical Education and Curricular Development

View full text Add to dashboard Cite

OBJECTIVES ChatGPT is an artificial intelligence model that can interpret free-text prompts and return detailed, human-like responses across a wide domain of subjects. This study evaluated the extent of the threat posed by ChatGPT to the validity of short-answer assessment problems used to examine pre-clerkship medical students in our undergraduate medical education program. METHODS Forty problems used in prior student assessments were retrieved and stratified by levels of Bloom's Taxonomy. Thirty of these problems were submitted to ChatGPT-3.5. For the remaining 10 problems, we retrieved past minimally passing student responses. Six tutors graded each of the 40 responses. Comparison of performance between student-generated and ChatGPT-generated answers aggregated as a whole and grouped by Bloom's levels of cognitive reasoning, was done using t-tests, ANOVA, Cronbach's alpha, and Cohen's d. Scores for ChatGPT-generated responses were also compared to historical class average performance. RESULTS ChatGPT-generated responses received a mean score of 3.29 out of 5 (n = 30, 95% CI 2.93-3.65) compared to 2.38 for a group of students meeting minimum passing marks (n = 10, 95% CI 1.94-2.82), representing higher performance ( P = .008, η2 = 0.169), but was outperformed by historical class average scores on the same 30 problems (mean 3.67, P = .018) when including all past responses regardless of student performance level. There was no statistically significant trend in performance across domains of Bloom's Taxonomy. CONCLUSION While ChatGPT was able to pass short answer assessment problems spanning the pre-clerkship curriculum, it outperformed only underperforming students. We remark that tutors in several cases were convinced that ChatGPT-produced responses were produced by students. Risks to assessment validity include uncertainty in identifying struggling students and inability to intervene in a timely manner. The performance of ChatGPT on problems requiring increasing demands of cognitive reasoning warrants further research.

show abstract

Section: Morjaria Et Almentioning

confidence: 99%

Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program

Morjaria,

Burns,

Bracken

et al. 2023

Journal of Medical Education and Curricular Development

View full text Add to dashboard Cite

show abstract

“…Members of our editorial team are also engaged with colleagues throughout the field of academic medicine and academic publishing to study, assess, and address emerging ethical questions related to scientific integrity and misconduct in the new era of GenAI. 17 Many of these ideas resonate with the recommendations advanced in a recent editors' statement 18 on the responsible use of GenAI technologies in scholarly journal publishing. Those editors recommended that large language model GPT or other GenAI tools not be included as authors and that authors should be fully transparent about their use of AI tools.…”

mentioning

confidence: 89%

“…Working in parallel with our authors, the full editorial team of Academic Medicine committed to ongoing review and revision of the policies and practices of our journal “to align with the academic standards of our field.” 13 Considerations that we have begun to work through are wide-ranging, related to the integrity and accessibility of data sets, strengthened review procedures, appropriate and selective use of AI detection tools in evaluating submissions, stricter requirements for authors and reviewers, safeguards with editorial management and publisher software, added monitoring processes, and potential consequences for AI-generated submissions by authors who failed to disclose their use of these tools. Members of our editorial team are also engaged with colleagues throughout the field of academic medicine and academic publishing to study, assess, and address emerging ethical questions related to scientific integrity and misconduct in the new era of GenAI 17 …”

mentioning

confidence: 99%

Addressing the Novel Implications of Generative AI for Academic Publishing, Education, and Research

Roberts

2024

Acad Med

View full text Add to dashboard Cite

“…A clear alignment fosters more robust learning environments that lead to desired learning outcomes. Additionally, as adult educators engage in the ethical questions of AI use in learning environments they will help to create guidelines that protect learner privacy, limit data collection, and ensure data security (Masters, 2023).…”

Section: Ai At the Intersection Of Healthcare Professionals And Patientsmentioning

confidence: 99%

Incorporation of artificial intelligence in healthcare professions and patient education for fostering effective patient care

Mucci,

Green,

Hill

2024

New Dir Adult Contin Educ

View full text Add to dashboard Cite

Artificial intelligence (AI) influences many aspects of modern life and has multiple applications in the delivery of healthcare. AI is designed to mimic human capabilities including pattern recognition, data analysis, and decision‐making and perform tasks more efficiently. It is capable of detecting patterns in large datasets that might elude human beings. AI has potential to improve patient care, patient safety, disease diagnosis and treatment; public health; health research; health administration; and the daily work of health professionals. In 2019, the US National Academy of Medicine emphasized the need for “physicians, nurses, and other clinicians, data scientists, health care administrators, public health officials, policy makers, regulators, purchasers of health care services, and patients to understand the basic concepts, current state of the art, and future implications of the revolution in AI and machine learning.” Therefore, it is incumbent on adult educators to incorporate AI training into health professions education and patient education.

show abstract

Ethical use of Artificial Intelligence in Health Professions Education: AMEE Guide No. 158

Cited by 94 publications

References 52 publications

Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program

Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program

Addressing the Novel Implications of Generative AI for Academic Publishing, Education, and Research

Incorporation of artificial intelligence in healthcare professions and patient education for fostering effective patient care

Contact Info

Product

Resources

About