Two‐Step Testing in Employee Selection: Is score inflation a problem?

Abstract

Unproctored Internet testing in employee selection has become increasingly popular over the past few years. However, there is a concern that cheating during unproctored administrations may influence the test results in terms of score inflation. The current research attempts to determine the extent of cheating on an unproctored Internet test of perceptual speed by analyzing data from a parallel version of the test administered in a proctored setting. Among a sample of 856 job applicants from nine European countries, there was no evidence that cheating occurred. In fact, evidence of an effect operating in the opposite direction was obtained. These results indicate that the use of unproctored Internet testing is appropriate for this type of test.

Cheating on Unproctored Internet Intelligence Tests: Strategies and Effects

“…With an effect size of .59 for overall test performance, the effect of cheating was substantial for the group of effective cheaters, supporting other studies that similarly detected cheating effects (Arthur et al, 2010;Nye et al, 2008). Moreover, it should be noted that the six highest scores were all found among the group of effective cheaters, implying that if a top-down selection strategy would be used, cheaters instead of honest candidates would be hired.…”

“…Absence of a human test administrator monitoring the test environment can easily lead to cheating and manipulated test scores (Ployhart, Weekley, Holtz, & Kemp, 2003). Although previous research (e.g., Nye et al, 2008) indicates that cheating in an unproctored test environment occurs, the effectiveness of different cheating strategies has not been established yet. Therefore, the aim of the present study was to investigate how people cheat and what effect cheating had on subtest scores and the total test score by explicitly inviting test takers to cheat on a UIT of cognitive ability.…”

“…having had access to test content prior to the assessment; Scott & Lezotte, 2012). Based on previous studies (Hausknecht, Halpert, Di Paolo, & Moriarty-Gerrard, 2007;Nye, Do, Drasgow, & Fine, 2008), it was expected that cheating would result in higher test scores (Hypothesis 1) and that this effect would depend on the number of cheating strategies used (Hypothesis 2). As different subtests require different cheating means and our test consisted of seven different subtests, we expected the largest cheating effect for candidates using a number of cheating strategies.…”

Mobile and Nonmobile Assessment in Organizations: Does Proctoring Make a Difference?

“…Increasing the time limit or comparing scores on the online version of the Wonderlic Personnel Test with an untimed cognitive ability assessment may yield different results. Previous research concerning speeded cognitive ability tests has focused primarily on retest effects across two test administrations (Arthur Jr. et al, 2010;Nye, Do, Drasgow, & Fine, 2008). This research design was not used in the present study; however, the time factor was necessary to simulate a high stakes assessment.…”

computerized adaptive testing in industrial and organizational psychology

“…Some possible solutions for limiting the detrimental effects of cheating in a high stakes unproctored Internet ability test are to use a speeded test (e.g., Arthur et al, 2009;Nye, Do, Drasgow, & Fine, 2008) or to have a two-step testing process where a confirmation test is used to verify the results of the first test (e.g., Beaty, Dawson, Fallaw, & Kantrowitz, 2009;Burke, van Someren, & Tatham, 2006). Nye et al (2008) did not detect any differences due to cheating when comparing proctored and unproctored test results for a perceptual speed test in a selection setting.…”