Using the script concordance test to assess clinical reasoning skills in undergraduate and postgraduate medicine

Abstract

A B S T R A C TThe script concordance test is a relatively new format of written assessment that is used to assess higherorder clinical reasoning and data interpretation skills in medicine. Candidates are presented with a clinical scenario, followed by the reveal of a new piece of information. The candidates are then asked to assess whether this additional information increases or decreases the probability or likelihood of a particular diagnostic, investigative, or management decision. To score these questions, the candidate's decision in each question is compared with that of a reference panel of expert clinicians. This review focuses on the development of quality script concordance Using the script concordance test to assess clinical reasoning skills in undergraduate and postgraduate medicine

Improving the validity of script concordance testing by optimising and balancing items

“…For Q3 in the sample questions, in thinking of the diagnosis of carcinoma of the colon, the student will not consider the additional information of a normal blood glucose or thyroid stimulating hormone (TSH) level. 2 Although it can be hard for examinees to simply 'disregard' previous hypotheticals and data in each question, they are reminded of the need to do this during the pre-examination briefing.…”

“…Each question under the same clinical scenario is intended to be independent of the other questions; that is, each additional piece of information is not influencing the probability of the diagnosis in the other questions. For Q3 in the sample questions, in thinking of the diagnosis of carcinoma of the colon, the student will not consider the additional information of a normal blood glucose or thyroid stimulating hormone (TSH) level . Although it can be hard for examinees to simply ‘disregard’ previous hypotheticals and data in each question, they are reminded of the need to do this during the pre‐examination briefing.…”

Accuracy of script concordance tests in fourth-year medical students

“…However, one of the limitations of using SCT in this context is the difficulty to give a clear and helpful feedback. Further, evidence supporting the validity of SCT scores with respect to examinees’ thought and response processes is still limited, 14 and potential weaknesses of SCT have recently been outlined 15 . See et al 16 analyzed SCT, and MCQ scores on pulmonary and critical care medicine tests in 16 fellows and 10 residents.…”

Commentary: expert responses in script concordance tests: a response process validity investigation

“…In our experience constructing SCT items, using simple and ill‐defined case scenarios to test core concepts in clinical reasoning in medicine, only 20–30% of SCT items have generally been discarded or modified because of discordance in response pattern amongst clinicians (i.e. extreme inconsistencies among experts) …”