The Operational Validity of a Video-Based Situational Judgment Test for Medical College Admissions: Illustrating the Importance of Matching Predictor and Criterion Construct Domains.

Lievens, Filip; Buyse, Tine; Sackett, Paul R.

doi:10.1037/0021-9010.90.3.442

Cited by 133 publications

(177 citation statements)

References 48 publications

Supporting

Mentioning

160

Contrasting

Unclassified

Order By: Relevance

“…3 SJTs have been shown to have significant validity in predicting future job performance and can offer incremental (additional) validity over other selection methods. [4][5][6] SJTs tend to produce smaller scoring differences between demographic groups than ability tests (eg based on ethnicity), 7 have high face validity 8 and are generally perceived positively by candidates. 5 SJTs have been recently developed and implemented for postgraduate medical selection in the UK.…”

Section: N B R I E F Educationmentioning

confidence: 99%

Could situational judgement tests be used for selection into dental foundation training?

Patterson

Ashworth²,

Mehra

et al. 2012

Br Dent J

View full text Add to dashboard Cite

Section: N B R I E F Educationmentioning

confidence: 99%

Could situational judgement tests be used for selection into dental foundation training?

Patterson

Ashworth²,

Mehra

et al. 2012

Br Dent J

View full text Add to dashboard Cite

“…Our SJT included items addressing practical everyday problems that a student would address both in and out of the classroom. Perhaps most similar to our use of the SJT was a study by Lievens, Buyse, and Sackett (2005) in which the authors described the actual use of a SJT for the selection of medical students in Belgium. They found it predicted grades incrementally over cognitive ability in an interpersonally oriented curriculum.…”

Section: Early Development and Validation Of The Noncognitive Measuresmentioning

confidence: 99%

Prediction of 4-year college student performance using cognitive and noncognitive predictors and the impact on demographic status of admitted students.

Schmitt

Keeney

Oswald

et al. 2009

Journal of Applied Psychology

136

119

View full text Add to dashboard Cite

This study was conducted to determine the validity of noncognitive and cognitive predictors of the performance of college students at the end of their 4th year in college. Results indicate that the primary predictors of cumulative college grade point average (GPA) were Scholastic Assessment Test/American College Testing Assessment (SAT/ACT) scores and high school GPA (HSGPA) though biographical data and situational judgment measures added incrementally to this prediction. SAT/ACT scores and HSGPA were collected and used in various ways by participating institutions in the admissions process while situational judgment measures and biodata were collected for research purposes only during the first few weeks of the participating students' freshman year. Alternative outcomes such as a self-report of performance on a range of student performance dimensions and a measure of organizational citizenship behavior, as well as class absenteeism, were best predicted by noncognitive measures. The racial composition of a student body selected with only cognitive measures or both cognitive and noncognitive measures under various levels of selectivity as well as the performance of students admitted under these scenarios is also reported. The authors concluded that both the biodata and situational judgment measures could be useful supplements to cognitive indexes of student potential in college admissions.Keywords: college admissions, selection, noncognitive measures, biodata, situational judgment As is true when organizations hire employees, colleges and universities seek to admit and recruit the best students. Just as the qualifications deemed to make a good employee vary across organizations or managers, so do the factors underlying notions about excellent student performance. In the educational context, these factors vary as a function of the university or admissions personnel who evaluate student credentials and performance. Traditionally, college admissions personnel use high school grade point averages (HSGPA), standardized tests of cognitive ability in the areas of verbal and mathematical skills (Scholastic Assessment Test/American College Testing Assessment, or SAT/ACT), and sometimes records of achievement in specific subject matter areas to assess student potential. Each factor provides unique information about the applicant. Letters of recommendation, essays, or interviews are being used increasingly by universities to complement these HSGPA and SAT/ACT scores. Schools vary widely in their assessment of the information contained in these supplemental materials. For example, while a reviewer at one school might assign a subjective rating to each component of the application, a reviewer at another school might form ratings of personal qualities (e.g., leadership) on the basis of a holistic review of the materials (Rigol, 2003). Clearly, any systematic and thorough processing of this information, especially when large numbers of applicants must be processed in a short period of time, places a heavy burden on admissions p...

show abstract

“…Various primary studies have examined whether SJTs significantly add to the prediction of job performance over cognitive ability, job knowledge, job experience, and personality (Chan and Schmitt, 2002;Clevenger et al, 2001;Lievens et al, 2005a;Oswald et al, 2004;Jones, 1997, 1999). Recently, McDaniel et al (2007) conducted a meta-analysis on the incremental validity of SJTs.…”

Section: Pr 374mentioning

confidence: 99%

Situational judgment tests: a review of recent research

2008

Self Cite

View full text Add to dashboard Cite

Purpose -The purpose of this paper is to give an empirically-based review of the strengths and weaknesses of situational judgment tests (SJTs). Design/methodology/approach -The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990-2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion-related validity, incremental validity, construct-related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects. Findings -Strengths of SJTs are that they show criterion-related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs might be prone to faking, practice, and coaching effects. There is also debate about what constructs are measured by SJTs. Research limitations/implications -Five avenues for future research are discussed: construct-related validity of SJTs, utility of SJTs vis-à -vis other predictors, impact of SJT features on validity and adverse impact, examination of alternative stimulus and response formats, and cross-cultural transportability of SJTs. Practical implications -Practitioners receive evidence-based information about the features, development, and strengths and weaknesses of SJTs. Originality/value -Apart from the USA, SJTs have not made strong inroads in selection practice in Europe and other parts of the world. This evidence-based paper might highlight the value of SJTs.

show abstract

The Operational Validity of a Video-Based Situational Judgment Test for Medical College Admissions: Illustrating the Importance of Matching Predictor and Criterion Construct Domains.

Cited by 133 publications

References 48 publications

Could situational judgement tests be used for selection into dental foundation training?

Could situational judgement tests be used for selection into dental foundation training?

Prediction of 4-year college student performance using cognitive and noncognitive predictors and the impact on demographic status of admitted students.

Situational judgment tests: a review of recent research

Contact Info

Product

Resources

About