This study is part of a trend of examining noncognitive predictors, for example, a situational judgment test (SJT), as supplements to cognitive predictors for making college admission decisions. The authors examined criterion data over multiple academic years and universities. The criterion domain was broadly conceptualized, including both cognitive and interpersonal domains. The sample consisted of 7,197 candidates of the Medical and Dental Studies Admission Exam in Belgium. Results confirmed the importance of cognitive predictors. A video-based SJT was differentially valid for predicting overall grade point average for different curricula. The SJT showed incremental validity over cognitively oriented measures for curricula that included interpersonal courses, but not for other curricula. The SJT became more valid through the years. This demonstrates the importance of carefully specifying predictor-criterion linkages and of differentiating both predictor and criterion constructs.

This study proposes a framework for examining the effects of retaking tests in operational selection settings. A central feature of this framework is the distinction between within-person and between-person retest effects. This framework is used to develop hypotheses about retest effects for exemplars of 3 types of tests (knowledge tests, cognitive ability tests, and situational judgment tests) and to test these hypotheses in a high stakes selection setting (admission to medical studies in Belgium). Analyses of within-person retest effects showed that mean scores of repeat test takers were one-third of a standard deviation higher for the knowledge test and situational judgment test and one-half of a standard deviation higher for the cognitive ability test. The validity coefficients for the knowledge test differed significantly depending on whether examinees' test scores on the first versus second administration were used, with the latter being more valid. Analyses of between-person retest effects on the prediction of academic performance showed that the same test score led to higher levels of performance for those passing on the first attempt than for those passing on the second attempt. The implications of these results are discussed in light of extant retesting practice.In employment settings, the Uniform Guidelines on Employee Selection Procedures (1978) state that organizations should provide a reasonable opportunity to test takers for retesting. Hence, most organizations in the private and public sector have installed retesting policies in promotion and hiring situations (e.g., Campbell,

This study fills a key gap in research on response instructions in situational judgment tests (SJTs). The authors examined whether the assumptions behind the differential effects of knowledge and behavioral tendency SJT response instructions hold in a large-scale high-stakes selection context (i.e., admission to medical college). Candidates (N = 2,184) were randomly assigned to a knowledge or behavioral tendency response instruction SJT, while SJT content was kept constant. Contrary to prior research in low-stakes settings, no meaningfully important differences were found between mean scores for the response instruction sets. Consistent with prior research, the SJT with knowledge instructions correlated more highly with cognitive ability than did the SJT with behavioral tendency instructions. Finally, no difference was found between the criterion-related validity of the SJTs under the two response instruction sets.

Cognitive tests are used to select students into dental school, yet cognitive predictors explain only part of the variance in academic performance. Therefore, interviews and personality tests are often used to measure noncognitive (e.g., interpersonal) characteristics. Recently, situational judgment tests (SJTs) have drawn attention since there is evidence that SJTs can be valid predictors in medical admission contexts. This study examines the validity of an SJT measuring interpersonal skills for predicting academic performance of dental students. Incremental validity over cognitive tests is also examined. In this study, 796 dental students who passed the admission exam for medical and dental students in Flanders, Belgium, and enrolled in one of the two Flemish dental schools were evaluated. Grade point average (GPA) in the ive years of dental studies served as the criterion. Corrected correlation between the cognitive tests of the admission exam and GPA equaled .38. Their validity dropped from .45 (year 1) to .18 (year 5). However, the validity of the SJT increased from .05 (year 1) to .20 (year 5). The SJT had incremental validity in year 5. Dental admissions committees that envision assessing a broad set of capabilities might consider using an SJT as a valuable supplement to cognitive tests. Future research needs to conirm these indings with job performance as another criterion.

Although the evidence for the use of situational judgment tests (SJTs) in high‐stakes testing has been generally promising, questions have been raised regarding the potential coachability of SJTs. This study reports the first examination of the effects of coaching on SJT scores in an operational high‐stakes setting. We contrast findings from a simple comparison of SJT scores for coached and uncoached participants (posttest only) with three different approaches to deal with the effects of self‐selection into coaching programs, namely using a pretest as a covariate and using two different forms of propensity score‐based matching using a wide range of variables as covariates. Coaching effects were estimated at about 0.5 SDs. The implications for the use of SJTs in high‐stakes settings and for coaching research in general are discussed.