Situational judgment tests: a review of recent research

Abstract

Purpose -The purpose of this paper is to give an empirically-based review of the strengths and weaknesses of situational judgment tests (SJTs). Design/methodology/approach -The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990-2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion-related validity, incremental validity, construct-related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects. Findings -Strengths of SJTs are that they show criterion-related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs might be prone to faking, practice, and coaching effects. There is also debate about what constructs are measured by SJTs. Research limitations/implications -Five avenues for future research are discussed: construct-related validity of SJTs, utility of SJTs vis-à -vis other predictors, impact of SJT features on validity and adverse impact, examination of alternative stimulus and response formats, and cross-cultural transportability of SJTs. Practical implications -Practitioners receive evidence-based information about the features, development, and strengths and weaknesses of SJTs. Originality/value -Apart from the USA, SJTs have not made strong inroads in selection practice in Europe and other parts of the world. This evidence-based paper might highlight the value of SJTs.

Could situational judgement tests be used for selection into dental foundation training?

“…Over the last 20 years, SJTs have become increasingly popular in large-scale selection, typically at the shortlisting stage. 3 SJTs have been shown to have significant validity in predicting future job performance and can offer incremental (additional) validity over other selection methods. [4][5][6] SJTs tend to produce smaller scoring differences between demographic groups than ability tests (eg based on ethnicity), 7 have high face validity 8 and are generally perceived positively by candidates.…”

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

“…The four aspects were tested in separate models because the small number of data points (i.e., 28) did not allow entering all four aspects in one model. The effect sizes were corrected for the reliability of the scoring method by dividing Cohen's d by coefficient alpha, since low reliability may obscure subgroup differences (Lievens et al 2008).…”

Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100

“…scenario content, response instructions and format) may differ depending on the level of job knowledge required. Research consistently shows that SJTs are usually found to be positively received by candidates compared to other selection tests such as cognitive ability and personality tests (Hausknecht et al 2004, Lievens & Sackett 2007, Lievens et al 2008. Practically, SJTs are difficult to design effectively, and significant expertise is required to build a reliable and valid SJT.…”