Bootstrap Inference on the Variance Component Functions in the Two-Way Random Effects Model with Interaction

Ye, Rendao; Ge, Wenting; Luo, Kun

doi:10.1007/s11424-020-9216-7

Cited by 2 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The conclusion is that alpha is not always close to the coefficient of generalizability: for a small number of multidimensional items, the estimation error in alpha may be sizable even if the subject sample is large. Confidence intervals for alpha in the two-way random model are available for normally distributed variables (McGraw and Wong 1996;Demetrashvili et al 2016), and bootstrap methods are developed for the variance components (Brennan 2007;Tong and Brennan 2007) and their ratio (Gilder et al 2007;Ye et al 2020). Still, it would be helpful for test makers in explorative research to have a simple guideline on the a priori minimum test length required for accurate estimation of the coefficient of generalizability with alpha.…”

Section: Alpha As An Estimate For the Coefficient Of Generalizabilitymentioning

confidence: 99%

A Test Can Have Multiple Reliabilities

Ellis

2021

Psychometrika

View full text Add to dashboard Cite

It is argued that the generalizability theory interpretation of coefficient alpha is important. In this interpretation, alpha is a slightly biased but consistent estimate for the coefficient of generalizability in a subjects x items design where both subjects and items are randomly sampled. This interpretation is based on the “domain sampling” true scores. It is argued that these true scores have a more solid empirical basis than the true scores of Lord and Novick (1968), which are based on “stochastic subjects” (Holland, 1990), while only a single observation is available for each within-subject distribution. Therefore, the generalizability interpretation of coefficient alpha is to be preferred, unless the true scores can be defined by a latent variable model that has undisputed empirical validity for the test and that is sufficiently restrictive to entail a consistent estimate of the reliability—as, for example, McDonald’s omega. If this model implies that the items are essentially tau-equivalent, both the generalizability and the reliability interpretation of alpha can be defensible.

show abstract