Assessment of Clinical Reasoning in an Environment of Uncertainty: A Script Concordance Test for Neonatal-Perinatal Medicine

Construct validity of script concordance testing: progression of scores from novices to experienced clinicians

“…The results from the current study suggest that clinical experience does have an effect on performance in SCT, providing further support for the construct validity of scores from this format of assessment. Whilst previous studies have reported similar results at post-graduate medical education stage and shown progression of scores as trainees advance through their training in Neonatology and Psychiatry, there have been no reports of progression of SCT scores when the same set of SCT items are used to compare the clinical reasoning ability of medical students, junior doctors and experienced clinicians 8 , 18 . A study from Brazil has shown progression of SCT scores from students in the pre-clinical phase to those in the clinical phase (51.6% to 63.4%) using 10 clinical cases.…”

“…Kazour examined interns (junior doctors in the first post-graduate year) using a set of 100 SCT questions in Psychiatry and found significant improvement in the interns' scores between the beginning and the end of their rotation 8 . A further study used SCT scenarios to assess the reasoning skills of paediatric residents and neonatal-perinatal medicine fellows (qualified specialists), and reported a significant difference between all training levels from Post-graduate Year 1 (PGY-1) to PGY-3 and between PGY-3 and fellows, with improvement of scores observed for each progressive level of medical training 18 . More recently, Subra administered an SCT assessment to post-graduate students in general practice and showed progression of clinical reasoning throughout the 3 years of training pathway especially in the first 18-months 19 .…”

Improving the validity of script concordance testing by optimising and balancing items

“…9,14 The construct validity of script concordance testing has been shown by various studies demonstrating progression of SCT scores from undergraduate medical students to postgraduate fellows in training. 8,15,[18][19][20] However, a recent study has suggested that the aggregate partial credit scoring method used in SCTs could be subjected to validity threats. 17 Lineberry et al showed that students who avoided selecting the extreme response options (i.e.…”

“…The reliability of SCT scores has been reported to be around a Cronbach's alpha value of 0.7–0.85 . The construct validity of script concordance testing has been shown by various studies demonstrating progression of SCT scores from undergraduate medical students to postgraduate fellows in training …”