Accuracy of script concordance tests in fourth-year medical students

Abstract

ObjectivesThis investigation aimed to determine the validity of script concordance test (SCT), compared with clinical-case-related short-answer management problems (SAMP), in fourth-year medical students. MethodsThis retrospective study was conducted at the Medical School of Lille University. Cardiology and gynecology examinations both included 3 SCT and 2 clinical-case-related SAMP. Final score did not include SCT results, and was out of 20 points. The passing score was ≥10/20. Wilcoxon and McNemar tests were used to compare quantitative and qualitative variables, respectively. Correlation between scores was also analyzed. ResultsA total of 519 and 521 students completed SAMP and SCT in cardiology and gynecology, respectively. Cardiology score was significantly higher in SCT than SAMP (mean ± SD 13.5±2.4 versus 11.4±2.6, Wilcoxon test, p<0.001). In gynecology, SCT score was significantly lower than SAMP score (10.8±2.6 versus 11.4±2.7, Wilcoxon test, p=0.001). SCT and SAMP scores were significantly correlated (p <0.05, Pearson’s correlation). However, percentage of students with SCT score ≥ 10/20 was similar among those who passed or failed cardiology (327 of 359 (91%) vs 146 of 160 (91%), χ2=0.004, df =1, p=0.952), or gynecology (274 of 379 (65%) vs 84 of 142 (59%), χ2=1.614, df=1, p=0.204) SAMP test. Cronbach alpha coefficient was 0.31 and 0.92 for all SCT and SAMP, respectively. Conclusions Although significantly correlated, the scores obtained in SCT and SAMP were significantly different in fourth-year medical students. These findings suggest that SCT should not be used for summative purposes in fourth-year medical students.

The Impact of Script Concordance Testing on Clinical Decision-Making in Paramedic Education

“…As healthcare becomes increasingly complex, developing and assessing clinical reasoning skills becomes critical. The SCT could serve as an effective method to achieve this, complementing traditional assessment tools and providing a more nuanced understanding of a student's learning progress [34][35][36][37][38][39][40][41][42][43].…”

Prueba de concordancia de guiones para entrenar el razonamiento clínico en estudiantes de fonoaudiología

“…Un estudio con carácter nacional implementado en Francia y dirigido a medicina interna, analizó la correlación de los puntajes obtenidos en tareas de RC a través de pruebas de selección múltiple y scripts, determinando una elevada asociación entre ambos (Cohen-Aubart et al, 2021). Otra investigación centrada en cuarto año de medicina, describió heterogeneidad en los puntajes de scripts cuando se compararon con la resolución de situaciones clínicas breves, sugiriendo utilizarlos solo con fines formativos (Nseir, Elkalioubie, Deruelle, Lacroix y Gosset, 2017). Dicha observación se asimila a recomendaciones previas (Arceo y Durante, 2013).…”

Development and psychometrics of script concordance test (SCT) in midwifery

“…This study showed relatively high internal reliability coefficient for this test (.74). A good test reliability for SCT is 0.7 to 0.8 (18, 26, 29), and different studies have reported different internal consistency coefficients as Cronbach's alpha of 0.78 (1), 0.745 (23), 0.73 (32), .85 (30), 0.79 (21), 0.73 (31), .8 (28), 0.9 (19). Andrew et al stated that based on SCT 3 or 5 test options, Cronbach's alpha coefficient varies from 0.78 to 0.68 (33).…”

“…SCT is used in various fields of medical education (13, 18-23), nevertheless, to our knowledge, its application in the field of midwifery training has not still been investigated or published in English or Persian languages. However, an article was published in French language about application of SCT in midwifery (24, 25).…”