Standard errors of two‐level scalability coefficients

Abstract

For the construction of tests and questionnaires that require multiple raters (e.g., a child behaviour checklist completed by both parents) a novel ordinal scaling technique is currently being further developed, called two-level Mokken scale analysis. The technique uses within-rater and between-rater coefficients to assess the scalability of the test. These coefficients are generalizations of Mokken's scalability coefficients. In this paper we derived standard errors for the two-level coefficients and for their ratios. The coefficients, the estimates, the estimated standard errors and the software implementation are discussed and illustrated using a real-data example, and a small-scale simulation study demonstrates the accuracy of the estimates.

Bias of Two-Level Scalability Coefficients and Their Standard Errors

“…Computation of the scalability coefficients and their standard errors by means of the delta method only assumes that the item scores follow a multinomial distribution with varying multinomial parameters across subjects (Koopman et al, in press). The cluster bootstrap assumes that data between subjects are independent.…”

“…131-152). Recently, Koopman et al (in press) applied the delta method to derive standard errors for two-level scalability coefficients. Let n be a vector of order ( m + 1 ) I containing the frequencies of all possible item-score patterns, each pattern taking the form n 1 2 … I x 1 x 2 … x I .…”

“…The covariance matrix of the scalability coefficients can be estimated as V ^ g ( n ) by using the sample estimates for G and V n . For two-level scalability coefficients, Koopman et al in press derived matrix G in Equation 5. Because the derivations are rather cumbersome and lengthy, they are omitted here.…”

“…Because the derivations are rather cumbersome and lengthy, they are omitted here. The interested reader is referred to Koopman et al in press. The estimated delta-method standard errors S E d ( H ) are obtained by taking the diagonal of ( V ^ g ( n ) ) 1 / 2 .…”

“…A more detailed description of the measurement models and the two-level coefficients is provided below. Crisan, Van de Pol, and Van der Ark (2016) generalized the two-level scalability coefficients for dichotomous items to polytomous items, and Koopman, Zijlstra, and Van der Ark (in press) derived standard errors for the estimated two-level scalability coefficients using the delta method (e.g., Agresti, 2012, pp. 577-581; Sen & Singer, 1993, pp.…”