Item response theory scoring and the detection of curvilinear relationships.

Abstract

Psychologists are increasingly positing theories of behavior that suggest psychological constructs are curvilinearly related to outcomes. However, results from empirical tests for such curvilinear relations have been mixed. We propose that correctly identifying the response process underlying responses to measures is important for the accuracy of these tests. Indeed, past research has indicated that item responses to many self-report measures follow an ideal point response process-wherein respondents agree only to items that reflect their own standing on the measured variable-as opposed to a dominance process, wherein stronger agreement, regardless of item content, is always indicative of higher standing on the construct. We test whether item response theory (IRT) scoring appropriate for the underlying response process to self-report measures results in more accurate tests for curvilinearity. In 2 simulation studies, we show that, regardless of the underlying response process used to generate the data, using the traditional sum-score generally results in high Type 1 error rates or low power for detecting curvilinearity, depending on the distribution of item locations. With few exceptions, appropriate power and Type 1 error rates are achieved when dominance-based and ideal point-based IRT scoring are correctly used to score dominance and ideal point response data, respectively. We conclude that (a) researchers should be theory-guided when hypothesizing and testing for curvilinear relations; (b) correctly identifying whether responses follow an ideal point versus dominance process, particularly when items are not extreme is critical; and (c) IRT model-based scoring is crucial for accurate tests of curvilinearity. (PsycINFO Database Record

Personality traits and maladaptivity: Unipolarity versus bipolarity

“…Personality trait domains generally are conceptualized in terms of variance shared by facets; however, facets themselves have unique variance, which may be accounted for by other personality domains (e.g., Bäckstr€ om, Larsson, & Maddux, 2009) or unique constructs. Finally, recent research (Carter et al, 2016) has demonstrated an advantage for ideal point item response modeling for detecting curvilinear relationships between traits and functioning; however, such methods require larger sample sizes and, based on previous results (Carter, Dalal, Guan, LoPilato, & Withrow, 2017;Carter et al, 2016), such scoring methods alone likely would not have grossly altered the present findings.…”

Prioritization of the Skills to Be Mastered for the Daily Jobs of Japanese Dental Hygienists

“…In this study, we applied item response theory (IRT), structural equation modeling (SEM), and logistic regression analysis to infer the required proficiency levels of the daily jobs of Japanese dental hygienists and the relationship between their daily jobs and motivation to work. e benefits of IRT include comprehensive analyses, a reduction in measurement errors, the ability to create tests or questionnaires, meaningful scaling of latent variables, objective calibrations and equating, evaluations of tests or questionnaires and item biases, high accuracy in assessments, and evaluations of models and person-fit models [16]. IRT refers to a family of mathematical models that attempt to explain the relationships between latent traits and their manifestations.…”

Aristotle's golden mean and the importance of bipolarity for personality models: A commentary on “Personality traits and maladaptivity: Unipolarity versus bipolarity”

“…A key analytical advance in the article by Williams and Simms () is the examination of predictive bipolarity—or the nonlinear relations between the traits and outcomes. Nonetheless, even this effort was likely limited in their study since methodological work has suggested that ideal point scoring, as opposed to the typical dominance methods (e.g., total score, factor scores) that they used, is necessary for fully uncovering predictive bipolarity when it is present (Carter, Dalal, Guan, LoPilato, & Withrow, ). Future work should continue to examine how the response process, including ideal point scoring, influences findings for psychometric and predictive bipolarity.…”