Context Situational judgement tests (SJTs) are widely used to evaluate ‘non‐academic’ abilities in medical applicants. However, there is a lack of understanding of how their predictive validity may vary across contexts. We conducted a systematic review and meta‐analysis to synthesise existing evidence relating to the validity of such tools for predicting outcomes relevant to interpersonal workplace performance. Methods Searches were conducted in relevant databases to June 2019. Study quality and risk of bias were assessed using the Quality In Prognosis Studies (QUIPS) tool. Results were pooled using random effects meta‐analysis and meta‐regressions. Results Initially, 470 articles were identified, 218 title or abstracts were reviewed, and 44 full text articles were assessed with 30 studies meeting the final inclusion criteria and were judged, overall, to be at moderate risk of bias. Of these, 26 reported correlation coefficients relating to validity, with a pooled estimate of 0.32 (95% confidence interval 0.26 to 0.39, P < .0001). Considerable heterogeneity was observed (I2 = 96.5%) with the largest validity coefficients tending to be observed for postgraduate, rather than undergraduate, selection studies (β = 0.23, 0.11 to 0.36, P < .001). The correction of validity coefficients for attenuation was also independently associated with larger effects (β = 0.13, 0.03 to 0.23, P = .01). No significant associations with test medium (video vs text format), cross‐sectional study design, or period of assessment (one‐off vs longer‐term) were observed. Where reported, the scores generally demonstrated incremental predictive validity, over and above tests of knowledge and cognitive ability. Conclusions The use of SJTs in medical selection is supported by the evidence. The observed trend relating to training stage requires investigation. Further research should focus on developing robust criterion‐relevant outcome measures that, ideally, capture interpersonal aspects of typical workplace performance. This will facilitate additional work identifying the optimal place of SJTs within particular selection contexts and further enhancing their effectiveness.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.