ChatGPT in Occupational Medicine: A Comparative Study with Human Experts

Abstract

The objective of this study is to evaluate ChatGPT’s accuracy and reliability in answering complex medical questions related to occupational health and explore the implications and limitations of AI in occupational health medicine. The study also provides recommendations for future research in this area and informs decision-makers about AI’s impact on healthcare. A group of physicians was enlisted to create a dataset of questions and answers on Italian occupational medicine legislation. The physicians were divided into two teams, and each team member was assigned a different subject area. ChatGPT was used to generate answers for each question, with/without legislative context. The two teams then evaluated human and AI-generated answers blind, with each group reviewing the other group’s work. Occupational physicians outperformed ChatGPT in generating accurate questions on a 5-point Likert score, while the answers provided by ChatGPT with access to legislative texts were comparable to those of professional doctors. Still, we found that users tend to prefer answers generated by humans, indicating that while ChatGPT is useful, users still value the opinions of occupational medicine professionals.

Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases

“…[41] Sharing such awed images and illustrations to non-cardiac experts, like medical students, nurses, or laypersons, could unintentionally generate or intensify misinformation, a concern exacerbated by automation biases. This highlights the need for caution in using AI-tools for didactic purposes, particularly in sensitive elds like healthcare education [42][43][44][45][46][47].…”

Toward Clinical Generative AI: Conceptual Framework

“…In this paradigm, AI-generated diagnoses or treatment plans are evaluated by a panel of medical experts, with the consensus among these experts on the validity of the AI’s recommendations serving as a measure of their reliability. This paradigm is particularly useful in assessing the AI’s performance in complex cases in which human expertise is invaluable, ranging from the psychiatric field in dealing with issues such as suicide risk assessment [ 34 ] to occupational medicine [ 35 ]; oncology, with the management of malignancies [ 36 ]; and complex surgical procedures such as bariatric surgery [ 37 ].…”

Measuring the Effectiveness of AI Tools in Clinical Research and Writing: A Case Study in Healthcare

“…In November 2022, OpenAI unveiled ChatGPT, marking a watershed moment in AI-human interactions. This powerful AI model has been used in a variety of fields, including medicine and healthcare [1][2][3][4][5][6][7][8][9], management [10][11][12], scientific writing and language editing services [13][14][15][16], tourism information services [17], education [18,19], and others [20][21][22]. Its ability to generate human-like text based on prompts has opened up new opportunities for reshaping healthcare practices, improving patient care, and altering professional-patient interactions [23,24].…”