Script concordance test in medical schools in Brazil: possibilities and limitations

Abstract

CONTEXT AND OBJECTIVE: Routine use of the script concordance test (SCT) is not common in Brazilian universities. This study aimed to analyze application of the SCT in the medical school of a Brazilian university. DESIGN AND SETTING: Quantitative, analytical and descriptive study in the medical school of a Brazilian university. METHODS: A total of 159/550 students participated. The test comprised ten clinical cases within internal medicine, with five items per case, rated on a five-point Likert scale. The test was scored in accordance with a marking key that had been validated by a reference panel. RESULTS:In the pre-clinical and clinical phases, the mean scores were 51.6% and 63.4% of the maximum possible scores, respectively. Comparison of the means of the responses among all the years showed that there were significant differences in 40% of the items. The panel marked all the possible answers in five items, while in one item, all the panelists marked a single answer. Cronbach's alpha was 0.64. The results indicated that the more senior students performed better. Construction of an SCT with discriminative questions was not easy. The low reliability index may have occurred due to: a) problems with the construction of the questions; b) limitations of the reference panel; and/or c) the scoring key. CONCLUSION:This instrument is very difficult to construct, apply and correct. These difficulties may make application of an SCT as an assessment method unfeasible in units with limited resources. RESUMO

Individual versus Team-Based responses to Script Concordance Test in Teaching Clinical Reasoning

“…Another possible explanation is the students training and familiarity with the test format. This point met agreement with Roberti et al [25] who suggested that the SCT di culties in construction of the test and scoring key might make its application unfeasible in units with limited resources [25,26]. Also, Lineberry et al 2019 [26] reported that SCT can be used as a guide for assessment for learning.…”

Construct validity of script concordance testing: progression of scores from novices to experienced clinicians

“…A study from Brazil has shown progression of SCT scores from students in the pre-clinical phase to those in the clinical phase (51.6% to 63.4%) using 10 clinical cases. However, the authors concluded that the implementation of this exam format is difficult in under-resourced institutions and have not followed up on these findings 28 …”

Multicentered Study Evaluating Pharmacy Students’ Perception of Palliative Care and Clinical Reasoning Using Script Concordance Testing

“…19 The SCT has been introduced and evaluated in the medicine curricula to assess clinical reasoning for topics in undergraduate and postgraduate didactic and clinical education. [20][21][22][23][24][25][26][27][28][29][30][31] The SCT template has also been used to develop other assessment tools such as the ethics SCT to assess reasoning abilities on ethical dilemmas in high-cognitive level 28,29 or the evolving SCT where, as new information is presented at each stage, the clinical decision-making becomes increasingly clear. 22 The SCT has been combined with other clinical reasoning assessments such as the think aloud to enhance the assessment by allowing the examinees to provide justification for their responses that would have been considered incorrect and identify questions that are frequently misinterpreted.…”