The Fakeability of Personality Measurement with Graded Paired Comparisons

Abstract

This study compares the faking resistance of Likert scales and graded paired comparisons (GPCs) analyzed with Thurstonian IRT models. We analyzed whether GPCs are more resistant to faking than Likert scales by exhibiting lower score inflation and better recovery of applicants’ true (i.e., honest) trait scores. A total of $$N=573$$ N = 573 participants completed either the Likert or GPC version of a personality questionnaire first honestly and then in an applicant scenario. Results show that participants were able to increase their scores in both the Likert and GPC format, though their score inflation was smaller in the GPC than the Likert format. However, GPCs did not exhibit higher honest–faking correlations than Likert scales; under certain conditions, we even observed negative associations. These results challenge mean score inflation as the dominant paradigm for judging the utility of forced-choice questionnaires in high-stakes situations. Even if forced-choice factor scores are less inflated, their ability to recover true trait standings in high-stakes situations might be lower compared with Likert scales. Moreover, in the GPC format, faking effects correlated almost perfectly with the social desirability differences of the corresponding statements, highlighting the importance of matching statements equal in social desirability when constructing forced-choice questionnaires.