The inter-item standard deviation (ISD): An index that discriminates between conscientious and random responders

Abstract

Although random responding is prevalent and increases Type II errors, most psychologists avoid trying to identify it because the means to do so are extremely limited. We propose the inter-item standard deviation (ISD), a statistical index of response variance, is suited for this task. We hypothesized that random responders produce large ISDs because they respond to items all over a measure's response range, whereas conscientious responders produce small ISDs because they respond to items more consistently. We administered a questionnaire containing the NEO-FFI-3 and an embedded validity scale to 134 university students. Another 134 responders were created using a random number generator. For all 268 responders, the ISD was calculated for each of the NEO-FFI-3 0 s five subscales and an aggregated ISD was calculated by averaging the five ISD indexes. Results showed that (1) random responders produce significantly larger ISDs than conscientious responders, (2) the ISDs were strongly correlated with the embedded validity scale and with one another, and (3) the ISDs correctly identified responders with greater than 80% classification accuracy. The mean ISD yielded greater than 95% classification accuracy. This study shows that responders can be identified by quantifying inter-item response variance.

Believing is doing: Emotion regulation beliefs are associated with emotion regulation behavioral choices and subjective well-being

“…An additional nine were eliminated due to their low CRS scores, which identified them as random responders (i.e., cases with CRS sum score ≤ 3). This constitutes a random responding rate of 7.69% and is consistent with random responding rates found in other undergraduate samples surveyed with the CRS (e.g., Marjanovic, Holden, Struthers, Cribbie, & Greenglass, 2015; Marjanovic et al, 2014) and similar validity measures (see Meade & Craig, 2012). (Note that all statistics and indices were strengthened and improved after the deletion of the nine identified random responders).…”

Methods for the detection of carelessly invalid responses in survey data

“…This technique, proposed and tested by Marjanovic, Holden, Struthers, Cribbie, and Greenglass (2015), measures how much an individual strays from their own personal midpoint across a set of scale items. This measure is accomplished through the following formula, from Marjanovic et al (2015):…”

“…It is proposed by Marjanovic et al (2015) that higher values on ISD are more indicative of random responding. It is important here to continue to distinguish this older specific concept of random responding from the more modern and general concept of C/IE responding.…”

“…It is important here to continue to distinguish this older specific concept of random responding from the more modern and general concept of C/IE responding. Marjanovic et al (2015) tested the effectiveness of ISD on detecting pure random responses (responses drawn from a uniform random distribution, independently on each item), something that it is fairly easy to do with almost any of these metrics. However, scores on this technique do not appear to be linearly related to randomness, per the example above.…”

“…Person 1 would receive the best possible score on ISD, and be classified by Marjanovic et al (2015) as conscientious, as opposed to random. This is a fair assessment as long as the dichotomy is between conscientious and random, but hardly useful if the dichotomy is between conscientious and C/IE.…”