On the Practical Consequences of Misfit in Mokken Scaling

Abstract

Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thumb. Using simulated data, they concluded that items with low scalability had some influence on the reliability of test scores, person ordering and selection, and criterion-related validity estimates. Removing the misfitting items from the scale had, in general, a small effect on the outcomes. Although important outcome variables were fairly robust against scale violations in some conditions, authors conclude that researchers should not rely exclusively on algorithms allowing automatic selection of items. In particular, content validity must be taken into account to build sensible psychometric instruments.

A validation of the religious and spiritual struggles scale among young people living with HIV in Zimbabwe: Mokken scale analysis and exploratory factor analysis

“…The Loevinger’s scalability coefficients permit the assortment of items that measure the same latent trait in the Mokken scale from an item group. A Mokken scale is considered a weak scale when 0.3 ≤ H < 0.4, a medium scale when 0.4 ≤ H < 0.5, and a strong scale when H > = 0.5 ( Crişan et al, 2020 ).…”

Medical outcomes study social support survey (MOS-SSS) in patients with chronic disease: A psychometric assessment

“…To test the internal structure of the instrument, we first applied a non-parametric approach, Mokken scaling analysis (MSA) ( 125 ), that is a method focused on the psychometric properties of the observed score by analyzing the number of dimensions, the scaling of items and scores, local independence, and the monotonic item-score relationship ( 125 , 126 ), as these are characteristics that build the monotonic homogeneity model (MHM) ( 125 ). MSA does not require the assumptions of parametric analyses [e.g., structural equation modeling or item response theory; Crişan et al ( 127 )] and is a preliminary procedure for subsequent latent construct analysis ( 127 , 128 ). Additionally, this method was considered appropriate given the moderate sample size in each randomly drawn subsample and the small number of items in some of the MOS-SSS subscales.…”

“…Additionally, this method was considered appropriate given the moderate sample size in each randomly drawn subsample and the small number of items in some of the MOS-SSS subscales. Within the MSA, to determine the number of instrument scales, the automated item selection procedure (AISP) ( 125 , 126 ) was used with the normal search based on the increasing scalability of items grouped by the scalability coefficient H ( 127 ). The analysis was performed with the R program mokken ( 129 ).…”

A two-step, test-guided Mokken scale analysis, for nonclustered and clustered data

“…8) to get plausible values of the population coefficient H. The fit of Mokken models is investigated using several available methods, such as conditional association and manifest monotonicity (e.g., [8]). Items that show (severe) misfit may be adjusted or removed [41].…”

“…The fit of Mokken models is investigated using several available methods, such as conditional association and manifest monotonicity (e.g., [ 8 ]). Items that show (severe) misfit may be adjusted or removed [ 41 ]. Subsequently, reliability analysis may be performed or more strict measurement models (e.g., PIRT models) may be fitted.…”