Standard Errors of IRT Parameter Scale Transformation Coefficients: Comparison of Bootstrap Method, Delta Method, and Multiple Imputation Method

Abstract

The present study evaluated the multiple imputation method, a procedure that is similar to the one suggested by Li and Lissitz (2004), and compared the performance of this method with that of the bootstrap method and the delta method in obtaining the standard errors for the estimates of the parameter scale transformation coefficients in item response theory (IRT) equating in the context of the common-item nonequivalent groups design. Two different estimation procedures for the variancecovariance matrix of the IRT item parameter estimates, which were used in both the delta method and the multiple imputation method, were considered: empirical cross-product (XPD) and supplemented expectation maximization (SEM). The results of the analyses with simulated and real data indicate that the multiple imputation method generally produced very similar results to the bootstrap method and the delta method in most of the conditions. The differences between the estimated standard errors obtained by the methods using the XPD matrices and the SEM matrices were very small when the sample size was reasonably large. When the sample size was small, the methods using the XPD matrices appeared to yield slight upward bias for the standard errors of the IRT parameter scale transformation coefficients.

A note on computing Louis’ observed information matrix identity for IRT and cognitive diagnostic models

“…The relative execution time indicates that Louis’ formula always took less time in all respects than Oakes’ formula and ND methods either in absolute or relative terms. Finally, although the CP method was the fastest, its estimates have been found to be less accurate when used to compute standard error estimates (Paek & Cai, 2014; Zhang & Zhao, 2019).…”

Asymptotic Standard Errors of Parameter Scale Transformation Coefficients in Test Equating Under the Nominal Response Model

“…The R code can be made available upon request. For comparison, the multiple imputation method (Zhang & Zhao, 2019) was also applied to produce the standard errors for the two coefficients. The criterion standard errors were the empirical standard deviations of the estimated values of A and B calculated over 10,000 replications (Paek & Cai, 2014; Zhang & Zhao, 2019).…”

“…Despite the availability of the characteristic curve procedure for finding A and B , the approaches to estimating the standard errors for the estimates of A and B have not been developed and examined. There are at least three approaches to obtaining the standard errors for the estimates of A and B in test equating under the IRT models: the bootstrap method (Kolen & Brennan, 2014), the delta method (Andersson, 2018; Ogasawara, 2001; Zhang, 2020a), and the multiple imputation method (Zhang & Zhao, 2019). The bootstrap method is a resampling approach.…”

“…Therefore, it was not considered in the study. The multiple imputation method makes use of multiple sets of random imputations of the item parameter estimates to compute the standard errors of equating coefficients, which is significantly less computationally demanding than the bootstrap method (Zhang, 2020b; Zhang & Zhao, 2019). The delta method is an analytical approach, the application of which depends on the development of relatively complicated mathematical formulas for the standard error expressions (e.g., Andersson, 2018; Ogasawara, 2001; Zhang, 2020a).…”

“…However, provided that the mathematical formulas are derived, the delta method is much more efficient than both the bootstrap method and the multiple imputation method. More details about the comparison of the three standard error estimation procedures can be seen elsewhere (e.g., Kolen & Brennan, 2014; Zhang, 2020b; Zhang & Zhao, 2019).…”