Comparing Irt Properties of Different Likert Scale Lengths: a Case From Attitudinal Measurement on Physics Education Research

Abstract

Collapsing Likert scale length on attitudinal measurement toward learning and instruction can be an option by physics education researchers (PER) while interpreting categorical responses from participants. Psychometric evaluation of the different scale length, however, has inconclusive results to date. Item response theory (IRT) offers advanced information at the item level that has been approached in this simulation study to explore psychometric properties of five different Likert scale lengths. The total of 1000 artificial data were generated within five scale variations based on the underlying marginal distribution and inter item correlation matrix. Three parameters (item parameters, global fit indices, and test information functions (TIFs)) from Graded Response Model (GRM) were calculated to compare psychometric behaviors of the observed Likert scale length. Our findings demonstrated the discrimination indices among the five scale lengths remain constant. Global fit indices have no distinguishable information to differentiate the varying points. Based on item endorsement and TIFs, our simulation results confirmed previous works that more information will be raised by the greater number of Likert scale points. The implication intended by this study should provide a key recommendation for a case from attitudinal measurement on the PER community or even beyond the psychometric field.

Psychometric properties of the Cultural Intelligence Scale based on item response theory

“…They reported that ability estimates were good and stable under different conditions, and that the ability estimates became with higher accuracy when the sample size increased (500 and more). Other studies (Lee & Paek, 2014;Santoso et al, 2023) investigated the effect of different Likert scale response categories (3, 5, 7, 9, and 11 points) on psychometric properties within the IRT-GRM framework, and reported no differences among them as long as their items have good discrimination indices (>0.3). On the other side, Maydeu-Olivares et al (2009) reported that within IRT framework, increasing the number of response categories led to some improvement in precision of measurement, defined as the area under the TIF, however, it did not lead to increasing convergent validity, and it resulted in a worse fit of the model.…”