A Natural Language Processing–Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process: Simulator Development and Case Study

Furlan, Raffaello; Gatti, Mauro; Menè, Roberto; Shiffer, Dana; Marchiori, Chiara; Levra, Alessandro Giaj; Saturnino, Vincenzo; Brunetta, Enrico; Dipaola, Franca

doi:10.2196/24073

Cited by 20 publications

(21 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the adaptations that led to such learning opportunities are the clinical case discussions, role-playing of clinical cases, using the medical simulation center, and, to a lesser extent, the use of the web-based clinical simulated patient (DxR ® ). Furlan et al 25 and Kiesewetter et al 26 reported that clerkship students can benefit from similar web-based case discussion software in increasing their clinical reasoning skills, especially during crises times and impossibility of training on real patients. Furthermore, a scoping review by Park et al 27 showed that most clinical competencies could be learned online or in the virtual setting, which supports the usefulness of our implemented adaptations.…”

Section: Discussionmentioning

confidence: 99%

Adaptations of Clinical Teaching During the COVID-19 Pandemic: Perspectives of Medical Students and Faculty Members

Hassan¹,

El-Agroudy²,

Shehata³

et al. 2022

AMEP

View full text Add to dashboard Cite

Background The COVID-19 pandemic had serious implications on medical schools’ programs that necessitated lots of adaptations of teaching, learning, and assessment to guarantee continuity of education in medical schools. Our study aimed to evaluate perspectives of clerkship students and faculty members regarding clinical teaching adaptations implemented during the COVID-19 pandemic. Methods A descriptive, cross-sectional, survey-based study was conducted and targeted 5 th and 6 th year clerkship students and full- and part-time clinical faculty. The survey explored (1) perception of the degree of contribution of implemented adaptations to student achievement of expected clinical competencies, (2) degree of confidence regarding students’ achievement of expected clinical competencies through such adaptations, and (3) perception of the effect of implemented educational adaptations on students’ learning. Descriptive statistics were used, and statistical significance level was set at p < 0.05. Results The survey exhibited high internal consistency. Both students and faculty members felt that most of the adaptations had moderate to high contribution to student achievement of expected clinical competencies. On a 5-point scale, the highest score was given by faculty members to “Interpretation of investigations” (3.93±0.84) while the lowest scores were given by faculty members (3.10±1.21) and students (2.57±1.36) to “Performing clinical procedures”. Students and faculty members agreed that the adaptations had positive effect on students’ learning except for the statement “Students were able to easily monitor their academic progress” where students gave less scores than faculty members, with a statistically significance difference (p=0.029). Conclusion Students and faculty members had similar perspectives regarding the implemented adaptations and their impact and contribution to student learning and achievement of the basic clinical competencies. Both of them agreed on the need for and importance of the implemented adaptations. Our findings recommend such adaptations during the times of crises, which can be conducted through integrating online teaching with face-to-face teaching.

show abstract

Section: Discussionmentioning

confidence: 99%

Adaptations of Clinical Teaching During the COVID-19 Pandemic: Perspectives of Medical Students and Faculty Members

Hassan¹,

El-Agroudy²,

Shehata³

et al. 2022

AMEP

View full text Add to dashboard Cite

show abstract

“…This section provides a synthetic description of the main features underlying Hepius's diagnostic model, which is necessary for the full comprehension of the learning analytics. A detailed description of the program is provided elsewhere [23].…”

Section: Diagnostic Process Simulator Componentsmentioning

confidence: 99%

“…We recently developed a VPS, Hepius, which integrates ITS components [23] that address 2 main activities carried out by a physician when managing a patient: data gathering and data analysis. NLP techniques were used to mimic physician-patient interactions.…”

Section: Introductionmentioning

confidence: 99%

Learning Analytics Applied to Clinical Diagnostic Reasoning Using a Natural Language Processing–Based Virtual Patient Simulator: Case Study

Furlan¹,

Gatti²,

Menè³

et al. 2022

JMIR Med Educ

Self Cite

View full text Add to dashboard Cite

Background Virtual patient simulators (VPSs) log all users’ actions, thereby enabling the creation of a multidimensional representation of students’ medical knowledge. This representation can be used to create metrics providing teachers with valuable learning information. Objective The aim of this study is to describe the metrics we developed to analyze the clinical diagnostic reasoning of medical students, provide examples of their application, and preliminarily validate these metrics on a class of undergraduate medical students. The metrics are computed from the data obtained through a novel VPS embedding natural language processing techniques. Methods A total of 2 clinical case simulations (tests) were created to test our metrics. During each simulation, the students’ step-by-step actions were logged into the program database for offline analysis. The students’ performance was divided into seven dimensions: the identification of relevant information in the given clinical scenario, history taking, physical examination, medical test ordering, diagnostic hypothesis setting, binary analysis fulfillment, and final diagnosis setting. Sensitivity (percentage of relevant information found) and precision (percentage of correct actions performed) metrics were computed for each issue and combined into a harmonic mean (F1), thereby obtaining a single score evaluating the students’ performance. The 7 metrics were further grouped to reflect the students’ capability to collect and to analyze information to obtain an overall performance score. A methodological score was computed based on the discordance between the diagnostic pathway followed by students and the reference one previously defined by the teacher. In total, 25 students attending the fifth year of the School of Medicine at Humanitas University underwent test 1, which simulated a patient with dyspnea. Test 2 dealt with abdominal pain and was attended by 36 students on a different day. For validation, we assessed the Spearman rank correlation between the performance on these scores and the score obtained by each student in the hematology curricular examination. Results The mean overall scores were consistent between test 1 (mean 0.59, SD 0.05) and test 2 (mean 0.54, SD 0.12). For each student, the overall performance was achieved through a different contribution in collecting and analyzing information. Methodological scores highlighted discordances between the reference diagnostic pattern previously set by the teacher and the one pursued by the student. No significant correlation was found between the VPS scores and hematology examination scores. Conclusions Different components of the students’ diagnostic process may be disentangled and quantified by appropriate metrics applied to students’ actions recorded while addressing a virtual case. Such an approach may help teachers provide students with individualized feedback aimed at filling competence drawbacks and methodological inconsistencies. There was no correlation between the hematology curricular examination score and any of the proposed scores as these scores address different aspects of students’ medical knowledge.

show abstract

“…In recent years, deep learning and AI have been integrated into the development and application of ITS for health education purposes. For instance, Furlan et al ( 51 ) developed a virtual patient simulators (VPS) system that allowed students to gather clinical information from the patient's medical history, physical exam, and investigations and allowed them to formulate a differential diagnosis by using natural language. The VPS is also an ITS that provided real-time step-by-step feedback to the student and suggests specific topics the student has to review to fill in potential knowledge gaps.…”

Section: Recommendationsmentioning

confidence: 99%

A Framework of AI-Based Approaches to Improving eHealth Literacy and Combating Infodemic

Liu

Xiang

2021

Front. Public Health

View full text Add to dashboard Cite

The global COVID-19 pandemic has put everyone in an urgent need of accessing and comprehending health information online. Meanwhile, there has been vast amount of information/misinformation/disinformation generated over the Internet, particularly social media platforms, resulting in an infodemic. This public health crisis of COVID-19 pandemic has put each individual and the entire society in a test: what is the level of eHealth literacy is needed to seek accurate health information from online resources and to combat infodemic during a pandemic? This article aims to summarize the significances and challenges of improving eHealth literacy in both communicable (e.g., COVID-19) and non-communicable diseases [e.g., cancer, Alzheimer's disease, and cardiovascular diseases (CVDs)]. Also, this article will make our recommendations of a general framework of AI-based approaches to improving eHealth literacy and combating infodemic, including AI-augmented lifelong learning, AI-assisted translation, simplification, and summarization, and AI-based content filtering. This general framework of AI-based approaches to improving eHealth literacy and combating infodemic has the general advantage of matching the right online health information to the right people.

show abstract

A Natural Language Processing–Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process: Simulator Development and Case Study

Cited by 20 publications

References 46 publications

Adaptations of Clinical Teaching During the COVID-19 Pandemic: Perspectives of Medical Students and Faculty Members

Adaptations of Clinical Teaching During the COVID-19 Pandemic: Perspectives of Medical Students and Faculty Members

Learning Analytics Applied to Clinical Diagnostic Reasoning Using a Natural Language Processing–Based Virtual Patient Simulator: Case Study

A Framework of AI-Based Approaches to Improving eHealth Literacy and Combating Infodemic

Contact Info

Product

Resources

About