An Item-Level Analysis for Detecting Faking on Personality Tests: Appropriateness of Ideal Point Item Response Theory Models

Abstract

How to detect faking on personality measures has been investigated using various methods and procedures. As previous findings are mixed and rarely based on ideal point item response theory models, additional research is needed for further exploration. This study modeled the responses of personality tests using ideal point method across instructed faking and honest responding conditions. A sample of undergraduate students participated the within-subjects measures to examine how the item location parameter derived from the generalized graded unfolding model changed, and how individuals' perception about items changed when faked. The mean test scores of faking group was positively correlated to the magnitude of within-subjects score change. The item-level analysis revealed both conscientiousness items (18.8%) and neuroticism items (50.0%) appeared significant shifts on item parameters, suggesting that response pattern changed from honest to faking conditions. The direction of the change appeared both in positive and negative way, demonstrating that faking could increase or decrease personality factor scores. The results indicated that the changes of perceptions on items could be operated by faking, offering some support for the ideal point model to be an adequate measure for detecting faking. However, the findings of diagnostic accuracy analysis also implied that the appropriateness of ideal point models for detecting faking should be under consideration, also be used with caution. Implications, further research directions, and limitations are discussed.

The Effects of Aberrant Responding on Model-Fit Assuming Different Underlying Response Processes

“…Very few studies have examined the issue of aberrant data under an unfolding model context. Liu and Zhang (2020) investigated the appropriateness of ideal point models for detecting faking on personality measures (as opposed to honest responding). Results indicated that fitting the GGUM to the data under conditions where faking was present resulted in shifts in item location parameter estimates, demonstrating that faking could increase or decrease personality factor scores reflecting conscientiousness and neuroticism.…”

Challenges in Conducting Empirical Epidemiological Research with Truck and Bus Drivers in Diverse Settings in North America

“…Other statistical or methodological efforts to mitigate bias include adding assessments to measure social desirability itself, employing situational judgement tests to assess personality or behavioral tendencies, evaluating bias using item response theory techniques, or utilizing measures such as the Social Desirability Scale (SDS) [48][49][50]. However, a recent meta-analysis by Lanz et al, suggested the traditional SDS may provide little to no benefit in measuring either bias or a desirability trait [51].…”

Performance of Nonparametric Person-Fit Statistics with Unfolding versus Dominance Response Models

“…It is important for researchers to be able to detect varying types of aberrant responses and to be aware of the potential adverse effects they may have on datasets and analyses. Because of the limited research available on the use of person-fit statistics with ideal point response data, researchers have recommended that studies investigate their use with unfolding frameworks (Drasgow et al, 2010;Lee et al, 2014;J. Liu & Zhang, 2020;Polak et al, 2012;Tendeiro, 2017).…”