The Effects of Coaching on Situational Judgment Tests in High‐stakes Selection

Lievens, Filip; Buyse, Tine; Sackett, Paul R.; Connelly, Brian S.

doi:10.1111/j.1468-2389.2012.00599.x

Cited by 32 publications

(30 citation statements)

References 44 publications

(65 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One possible explanation for the larger effect in Lievens et al is that failure motivates students to attend more carefully to a coaching program, producing different effects than when coaching is prior to any test. The basis for the difference between the current finding and prior findings (Lievens et al, ) remains to be resolved.…”

Section: Discussioncontrasting

confidence: 90%

Effects of Organizationally Endorsed Coaching on Performance and Validity of Situational Judgment Tests

Stemig

Sackett

Lievens

2015

Int J Selection Assessment

Self Cite

View full text Add to dashboard Cite

There is growing interest in organizationally provided or organizationally endorsed coaching. However, little is known about the effects of such coaching on test scores in operational settings. This study reports on an examination of such a program in the context of the use of a situational judgment test (SJT) for medical school admissions. We examine the effects of multiple types of coaching methods on SJT scores and on their construct-related and predictive validities. Results suggest that (1) commercial coaching techniques may not be as effective as previously thought, whereas organizationally provided methods may be more effective, and that (2) the criterion-related validity of the SJT scores is not degraded by the availability of coaching. Generally, this study illustrates that concerns about potential unfairness of coaching can be countered by making effective coaching available to all examinees, in the form of organizationally endorsed coaching.

show abstract

Section: Discussioncontrasting

confidence: 90%

Effects of Organizationally Endorsed Coaching on Performance and Validity of Situational Judgment Tests

Stemig

Sackett

Lievens

2015

Int J Selection Assessment

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although there is extensive evidence confirming the predictive validity of SJTs in medical contexts (Lievens et al 2012;Lievens 2013;Patterson et al 2008Patterson et al , 2013Patterson et al , 2015a, there remains a relative shortage of evidence for the predictive validity of SJTs for performance for postgraduate trainees in their first role in clinical practice. Moreover, it is common practice in postgraduate medical settings internationally to combine measures of academic attainment and non-academic attributes in selection (Patterson et al 2015b(Patterson et al , 2016; however to date few researchers have empirically examined the complementary nature of these different selection methods in predicting in-role performance in practice (Prideaux et al 2011;Patterson et al 2016).…”

Section: Discussionmentioning

confidence: 99%

Evaluating the complementary roles of an SJT and academic assessment for entry into clinical practice

Cousans

Patterson

Edwards³

et al. 2017

Adv in Health Sci Educ

View full text Add to dashboard Cite

Although there is extensive evidence confirming the predictive validity of situational judgement tests (SJTs) -017-9755-4 performance. This is an important area of enquiry as despite it being common practice to use both types of methods within a selection system, there is currently no evidence that this approach translates into increased predictive validity of the selection system as a whole, over that achieved by the use of a single selection method. In this preliminary study, the majority of the range of scores achieved by successful applicants to the UK Foundation Programme provided a unique opportunity to address both of these areas of enquiry. Sampling targeted high ([80th percentile) and low (\20th percentile) scorers on the SJT. Supervisors rated 391 trainees' in-role performance, and incidence of remedial action was collected. SJT and academic performance scores correlated with supervisor ratings (r = .31 and .28, respectively). The relationship was stronger between the SJT and in-role performance for the low scoring group (r = .33, high scoring group r = .11), and between academic performance and in-role performance for the high scoring group (r = .29, low scoring group r = .11). Trainees with low SJT scores were almost five times more likely to receive remedial action. Results indicate that an SJT for entry into trainee physicians' first role in clinical practice has good predictive validity of supervisor-rated performance and incidence of remedial action. In addition, an SJT and a measure of academic performance appeared to be complementary to each other. These initial findings suggest that SJTs may be more predictive at the lower end of a scoring distribution, and academic attainment more predictive at the higher end.Adv in Health Sci Educ (2017) 22:401-413 DOI 10.1007/s10459

show abstract

“…First, although SJTs with behavior tendency items have higher correlations with personality than items eliciting knowledge, they may be easier to fake (Nguyen, Biderman, & McDaniel, ). Second, in a high‐stakes setting, Lievens, Buyse, Sackett, and Connelly () found an incremental effect ( SD = 0.5) between coaching and self‐test preparation across alternate forms of an interpersonal SJT in the context of a medical school admissions test. Third, challenges arise in relation to how to define someone as an expert in teamwork for the item authoring and scoring, as different situations may lend themselves to different kinds of expertise; hence, finding item writers to create a diverse range of plausible scenarios might be difficult (Zhuang et al, ).…”

Section: Considerations In Assessing Collaborative Problem Solvingmentioning

confidence: 99%

A Literature Review on Collaborative Problem Solving for College and Workforce Readiness

Oliveri

Lawless

Molloy

2017

ETS Research Report Series

View full text Add to dashboard Cite

The literature and the employee and workforce surveys rank collaborative problem solving (CPS) among the top 5 most critical skills necessary for success in college and the workforce. This paper provides a review of the literature on CPS and related terms, including a discussion of their definitions, importance to higher education and workforce success, and considerations relevant to their assessment. The goal is to discuss progress on CPS to date and help generate future research on CPS in relation to educational and workforce success.

show abstract

The Effects of Coaching on Situational Judgment Tests in High‐stakes Selection

Cited by 32 publications

References 44 publications

Effects of Organizationally Endorsed Coaching on Performance and Validity of Situational Judgment Tests

Effects of Organizationally Endorsed Coaching on Performance and Validity of Situational Judgment Tests

Evaluating the complementary roles of an SJT and academic assessment for entry into clinical practice

A Literature Review on Collaborative Problem Solving for College and Workforce Readiness

Contact Info

Product

Resources

About