Response Category Functioning on the Health Care Engagement Measure Using the Nominal Response Model

Abstract

As part of a scale development project, we fit a nominal response item response theory model to responses to the Health Care Engagement Measure (HEM). When using the original 5-point response format, categories were not ordered as intended for six of the 23 items. For the remaining, the category boundary discrimination between Categories 0 ( not at all true) and 1 ( a little bit true) was only weakly discriminating, suggesting uninformative categories. When the lowest two categories were collapsed, psychometric properties improved greatly. Category boundary discriminations within items, however, varied significantly. Specifically, higher response category distinctions, such as responding 3 ( very true) versus 2 ( mostly true) were considerably more discriminating than lower response category distinctions. Implications for HEM scoring and for improving measurement precision at lower levels of the construct are presented as is the unique role of the nominal response model in category analysis.

Item response theory.

“…These are a little tricky to interpret, but basically, they are the "impact" of the response on the trait level estimate. Stated differently, they are the weights by which to multiply a category response (i.e., to score) to obtain an optimally weighted composite; optimally weighted means a weighted composite that has a nonlinear correlation of 1.0 with the trait level estimate (see Reise et al, 2023). As such, they can also be thought of as reflecting category discrimination.…”

Analysis of distractors: an interpretation by the nominal response model of Enade 2017 items applied to the Degree in Mathematics

“…IRT has become one of the main tools for test analysis, especially to those from the sphere of educational evaluation for the last undergraduate years, as it allows for an accurate measurement of a latent trait in uni and multidimensional terms. However, the most common research studies and applications suggest the use of hit-or-miss patterns, enabling a dichotomous interpretation of the test's questions (Pinheiro;Costa;Cruz, 2010;Reise et al, 2023;Smith;Bendjilali, 2022). This interpretation assumes the application of a model for dichotomous items.…”

“…This enables a comprehensive analysis that considers both the totality and individual nuances. Within this approach, it is also feasible to estimate the discrimination parameters, which set apart the values associated with the most likely responses, as discussed by Reise et al (2023). The NRM provides a consistent foundation for an analysis that combines quantitative and qualitative elements, allowing detailed observations about the evolution of students and their skills.…”

Análise de distratores: uma interpretação pelo modelo de resposta nominal dos itens do Enade 2017 aplicado à Licenciatura em Matemática

“…A TRI tem se tornado umas das principais ferramentas para análises de testes, especialmente aqueles que pertencem à esfera da avaliação educacional nos últimos anos, uma vez que permite uma medição precisa de um traço latente em termos uni e multidimensionais. Contudo, as pesquisas e aplicações mais comumente encontradas sugerem o uso de padrões de acerto ou erro, o que permite uma interpretação dicotômica dos itens do teste (Pinheiro;Costa;Cruz, 2010;Reise et al, 2023;Smith;Bendjilali, 2022). Essa interpretação pressupõe a aplicação de um modelo para itens dicotômicos.…”