An Optimized Bayesian Hierarchical Two-Parameter Logistic Model for Small-Sample Item Calibration

Abstract

Accurate item calibration in models of item response theory (IRT) requires rather large samples. For instance, [Formula: see text] respondents are typically recommended for the two-parameter logistic (2PL) model. Hence, this model is considered a large-scale application, and its use in small-sample contexts is limited. Hierarchical Bayesian approaches are frequently proposed to reduce the sample size requirements of the 2PL. This study compared the small-sample performance of an optimized Bayesian hierarchical 2PL (H2PL) model to its standard inverse Wishart specification, its nonhierarchical counterpart, and both unweighted and weighted least squares estimators (ULSMV and WLSMV) in terms of sampling efficiency and accuracy of estimation of the item parameters and their variance components. To alleviate shortcomings of hierarchical models, the optimized H2PL (a) was reparametrized to simplify the sampling process, (b) a strategy was used to separate item parameter covariances and their variance components, and (c) the variance components were given Cauchy and exponential hyperprior distributions. Results show that when combining these elements in the optimized H2PL, accurate item parameter estimates and trait scores are obtained even in sample sizes as small as [Formula: see text]. This indicates that the 2PL can also be applied to smaller sample sizes encountered in practice. The results of this study are discussed in the context of a recently proposed multiple imputation method to account for item calibration error in trait estimation.

Bayesian Hierarchical Multidimensional Item Response Modeling of Small Sample, Sparse Data for Personalized Developmental Surveillance

“…This issue was adressed by using Bayesian hierarchical priors for the multivariate ability estimates and the discrimination parameter of the milestones. When using a Bayesian hierarchical prior, the hyperparameters of the first-level parameters (e.g., the children's abilities and the discrimination of the milestones) are not specified directly but are given prior distributions (König et al, 2020). This means that only the prior distributions for the hyperparameters are required to be specified, which alleviates the problem of having to specify informative priors when using a nonhierarchical model (König et al, 2020).…”

“…When using a Bayesian hierarchical prior, the hyperparameters of the first-level parameters (e.g., the children's abilities and the discrimination of the milestones) are not specified directly but are given prior distributions (König et al, 2020). This means that only the prior distributions for the hyperparameters are required to be specified, which alleviates the problem of having to specify informative priors when using a nonhierarchical model (König et al, 2020). Sheng (2012, p. 28) states that ''hierarchical priors offer flexibility to specify weakly informative priors on the hyperparameters'' and that using hierarchical priors allows for a more objective approach.…”

“…In addition to using the HMC NUTS sampler, a noncentered paramaterization was used for modeling the multidimensional ability distributions, which removes the dependency between the ability parameters u i and the global mean parameter m u (König et al, 2020). This can also help alleviate the computational problems associated with correlations between the parameters at different levels.…”

“…Bayesian hierarchical priors improve parameter estimation for small samples and sparse measurements by pooling information from parameters of the same type (König et al, 2020). In the Bayesian hierarchical prior, the parameters at the bottom level of the hierarchy (e.g., the ability parameters for the children or the item parameters of the milestones) are related to each other and are viewed as a sample from a common population distribution (Gelman et al, 2013).…”

“…The use of Bayesian MIRT models has been demonstrated on both simulated and real data examples (de la Torre & Patz, 2005;Sheng & Wikle, 2008), but they used large and complete samples. Sheng (2012) and König et al (2020) implemented Bayesian hierarchical univariate item response models for small samples, but both were simulation studies and they did not have sparse measurements. Only one example of a Bayesian hierarchical multidimensional item response model applied to small samples could be found in the literature (de la Torre & Hong, 2010); however, the smallest sample size used in the article was 500, which was part of a simulation study.…”