Evaluating the Distorting Effects of Inattentive Responding and Social Desirability on Self‐Report Scales in Creativity and the Arts

Abstract

Inattentiveness and social desirability might be particularly problematic for self-report scales in creativity and arts research. Respondents who are inattentive or who present themselves favorably will score highly on scales that yield positively skewed distributions and that assess socially valued constructs, such as scales measuring creativity and arts knowledge. A total of 204 undergraduates completed an online survey with several selfreport measures (the Creative Achievement Questionnaire, the Biographical Inventory of Creative Behavior, and the Aesthetic Fluency Scale). Many metrics of inattentiveness were included, such as directed response items, self-reported attentiveness, and scales to catch inconsistent and patterned responses. The Balanced Inventory of Desirable Responding measured facets of social desirability. We found high rates of inattentive responding. A latent class analysis indicated that around 25% of the sample was potentially inattentive, and inattentive respondents received higher scores on the creativity and arts scales. Excluding problematic participants increased the effect sizes of interest, consistent with the coarsening effect of careless participants. Social desirability, in contrast, had essentially no relationship with the creativity and arts scales. These findings suggest that social desirability is probably less of a problem than researchers fear, but that inattentiveness is an underappreciated threat. Some practical guidelines are suggested.

Relationship between sources of counseling self-efficacy and counseling self-efficacy among Malaysian school counselors

“…The methodology for data collection depends solely on self-reported measures that may encourage socially desirable responses (McKibben & Silvia, 2015).…”

Openness to experience and creativity: When does global citizenship matter?

“…More research with different populations from differing countries is warranted. In addition, while creativity check‐lists are widely used in creativity research and may provide information that is difficult to obtain via other measures, they are also not without weakness, as they may be prone to deficiencies such as participants' inattentiveness (McKibben & Silvia, ). However, a body of literature also finds self‐report creativity scales to be valid and reliable measures of creativity, predicting creative behaviour better than personality traits, and showing high convergent validity with other creativity measures (e.g., Silvia, Wigert, Reiter‐Palmon, & Kaufman, ).…”

An investigation of completion times on the Screener and Opioid Assessment for Patients with Pain – revised (SOAPP-R)

“…The use of response times to detect the aberrant answering of assessment items is well known 8–11,13–20,51–57. To our knowledge, no previous research has evaluated the utility of SOAPP-R completion times to predict ADB, despite the fact that respondent deception has been identified as a concern for this questionnaire 7.…”

“…Data on person-level confounding variables (eg, reading speed and cognitive skills) were unavailable; adjusting for these variables, as well as item-level variables (eg, item complexity and length), could also improve the signal-to-noise ratio of the data through a richer statistical model 28,52–56,58–61. Nevertheless, the direct use of the total amount of time taken on an assessment (ie, the raw completion time) had been found to have utility in previous work,12,37,62–65 including the identification of aberrant answering 8,21,57…”