Investigating Retesting Effects in a Personnel Selection Context

Dunlop, Patrick D.; Morrison, David; Cordery, John

doi:10.1111/j.1468-2389.2011.00549.x

Cited by 10 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They found larger retest (practice) effects for a cognitive ability measure than for the SJT (about 1/2 versus 1/3 of a standard deviation). In this study, unlike that of Dunlop et al (2011), the knowledge test showed retest effects more similar to the SJT than to the cognitive ability test. Lievens et al (2005) also found that the validity of the second SJT was higher than that of the first for those who retook the test.…”

Section: Context Effectscontrasting

confidence: 67%

Low-Fidelity Simulations

Weekley¹,

Hawkes

Guenole

et al. 2015

Annu. Rev. Organ. Psychol. Organ. Behav.

View full text Add to dashboard Cite

Low-fidelity simulations, which combine closed-ended response options with realistic depictions of key job tasks, have grown in popularity for several practical reasons. Research into low-fidelity simulations, which has focused primarily on text-based situational judgment tests (SJTs), has shown that such measures (a) can predict a wide range of criteria, doing so with moderate adverse impact, depending on the construct saturation of the items, (b) are no more susceptible to coaching/cheating/retesting effects than many other forms of assessment and far less susceptible than some, and (c) continue to struggle when used to measure a single construct. Practice, by contrast, has moved well beyond text-based SJTs to include multimedia SJTs, online assessment center exercises, and game-like assessments. Because these are perceived more favorably by applicants, the same research dedicated to text-based SJTs is needed to guide development of these newer forms of low-fidelity simulation.18.1

show abstract

Section: Context Effectscontrasting

confidence: 67%

Low-Fidelity Simulations

Weekley¹,

Hawkes

Guenole

et al. 2015

Annu. Rev. Organ. Psychol. Organ. Behav.

View full text Add to dashboard Cite

show abstract

“…Lievens et al, ). The significantly higher score for second‐time test takers ( d = 0.27) provides evidence for a small retest effect when using the standardized or dichotomous consensus scoring method, which corresponds to previous research on retest effects on SJTs (Dunlop, Morrison, & Cordery, ; Lievens et al, ). Retest effects could represent faking, but could also represent a practice effect or actual improvement in the relevant construct (Hooper et al, ).…”

Section: Discussionsupporting

confidence: 79%

Faking on a situational judgment test in a medical school selection setting: Effect of different scoring methods?

Leng

Stegers‐Jager

Born

et al. 2019

Int J Selection Assessment

View full text Add to dashboard Cite

We examined the occurrence of faking on a rating situational judgment test (SJT) by comparing SJT scores and response styles of the same individuals across two naturally occurring situations. An SJT for medical school selection was administered twice to the same group of applicants (N = 317) under low‐stakes (T1) and high‐stakes (T2) circumstances. The SJT was scored using three different methods that were differentially affected by response tendencies. Applicants used significantly more extreme responding on T2 than T1. Faking (higher SJT score on T2) was only observed for scoring methods that controlled for response tendencies. Scoring methods that do not control for response tendencies introduce systematic error into the SJT score, which may lead to inaccurate conclusions about the existence of faking.

show abstract

“…A plateau does not seem to be reached after four test administrations and further acceleration of mental processing might take place. This is of special interest, as many studies with cognitive ability tests focusing on score gains and their course over multiple test repetitions have found retest effects to reach a plateau somewhat earlier [7,107,108,109,110]. Thus, retest effects might be interpreted differently when RT is the outcome measure than when the total score is.…”

Section: Discussionmentioning

confidence: 99%

“…For this kind of cognitive ability tasks, outcomes are mostly reported as RTs. Though, it might be interesting to investigate the role of RT reduction in other cognitive domains as well, as differences between cognitive operations have been observed in score gains due to retesting [63,108,110]. In this regard, g loadings might explain differences in RT reduction.…”

Section: Discussionmentioning

confidence: 99%

Response Time Reduction Due to Retesting in Mental Speed Tests: A Meta-Analysis

2018

View full text Add to dashboard Cite

As retest effects in cognitive ability tests have been investigated by various primary and meta-analytic studies, most studies from this area focus on score gains as a result of retesting. To the best of our knowledge, no meta-analytic study has been reported that provides sizable estimates of response time (RT) reductions due to retesting. This multilevel meta-analysis focuses on mental speed tasks, for which outcome measures often consist of RTs. The size of RT reduction due to retesting in mental speed tasks for up to four test administrations was analyzed based on 36 studies including 49 samples and 212 outcomes for a total sample size of 21,810. Significant RT reductions were found, which increased with the number of test administrations, without reaching a plateau. Larger RT reductions were observed in more complex mental speed tasks compared to simple ones, whereas age and test-retest interval mostly did not moderate the size of the effect. Although a high heterogeneity of effects exists, retest effects were shown to occur for mental speed tasks regarding RT outcomes and should thus be more thoroughly accounted for in applied and research settings.

show abstract

Investigating Retesting Effects in a Personnel Selection Context

Cited by 10 publications

References 6 publications

Low-Fidelity Simulations

Low-Fidelity Simulations

Faking on a situational judgment test in a medical school selection setting: Effect of different scoring methods?

Response Time Reduction Due to Retesting in Mental Speed Tests: A Meta-Analysis

Contact Info

Product

Resources

About