A generalizability analysis of score consistency for the Balanced Inventory of Desirable Responding.

Abstract

Our goal in this investigation was to evaluate the reliability of scores from the Balanced Inventory of Desirable Responding (BIDR) more comprehensively than in prior research using a generalizability-theory framework based on both dichotomous and polytomous scoring of items. Generalizability coefficients accounting for specific-factor, transient, and random-response error ranged from .64 to .75 for the BIDR's Self-Deception Enhancement (SDE) and Impression Management (IM) subscale scores, and these values were systematically lower than corresponding alpha (.66 to .83) and 1-week test-retest (.78 to .86) coefficients. Polytomous scoring provided higher reliability than dichotomous scoring on nearly all indexes reported. Random-response (8%-17%) and specific factor error (11%-17%) exceeded transient error (3%-6%) for both subscales and scoring methods. Doubling the number of items on a single occasion provided greater improvements in generalizability (.76-.83) than aggregating scores across 2 administrations (.72-.81). Both scoring methods provided reasonably high indexes of consistency (φ coefficients≥.91) at cut scores on the IM scale for detecting faked responses when all sources of error were taken into account. Implications of these results for common uses of the BIDR are discussed.

Using G-Theory to Enhance Evidence of Reliability and Validity for Common Uses of the Paulhus Deception Scales

“…The persons by items by occasions design is linked to traditional applications of G-theory in which measures are viewed as being randomly rather than classically parallel. In keeping with tradition, investigators have conceptualized results in this way when applying G-theory to objectively scored measures (see, e.g., Christopherson, Helseth, & Lund, 2008; Nugent, 2006; Vispoel & Tao, 2013; Webb, Rowley, & Shavelson, 1988). However, G-coefficients for randomly parallel measures will share shortcomings of alpha coefficients in being conservative estimates of reliability because individual items will generally vary more from each other than will parallel splits or parallel forms.…”

Response Bias, Malingering, and Impression Management

“…month testÀretest reliability coefficient of .74 for the IM scale score Vispoel and Tao (2013). reported testÀretest reliability coefficients of .78 and .83 for dichotomous and polytomous scoring of SDE scale, as well as .83 and .86 for the IM scale, over a one-week interval.…”

“…Vispoel and Tao (2013) reported higher coefficients on all measures with a 7-point polytomous response scale than with a dichotomous response scale, suggesting that a polytomous scoring system should be used (Li & Bagger, 2007;Stőber, Dette, & Musch, 2002). Vispoel and Tao (2013) reported higher coefficients on all measures with a 7-point polytomous response scale than with a dichotomous response scale, suggesting that a polytomous scoring system should be used (Li & Bagger, 2007;Stőber, Dette, & Musch, 2002).…”

Personality Test Faking: Detection and Selection Rates

“…In the past, some have included unconscious behaviors that distort the truth to promote a positive impression, although most current researchers use definitions that imply or explicitly require intent (MacCann, Ziegler, & Roberts, 2011;Paulhus, 1984). Vispoel and Tao (2013) define socially desirable responding as a tendency (unconscious or willful) to respond to items to make appear good rather than be truthful. Hogan, Barrett, and Hogan (2007) defined impression management as a process in which one controls their behavior during social interaction, which includes responding to inventories.…”