A Novel Bootstrap Procedure for Assessing the Relationship between Class Size and Achievement

Abstract

There is on-going concern about the relationship between class size and achievement for children in their first years of schooling. The Institute of Education's class size project was set up to address this issue and began recruiting in the autumn of 1996. However, because of the non-normality of achievement measures, especially in mathematics, the results have hitherto been presented by using transformed achievement measures. This makes the interpretation difficult for non-statisticians. Ideally, the data would be modelled on the original scale and a bootstrap procedure used to ensure that inferences are robust to non-normality. However, the data are multilevel. In the paper we therefore propose a nonparametric residual bootstrap procedure that is suitable for multilevel models, show that it is consistent and present a simulation study which demonstrates its potential to yield substantial reductions in the difference between nominal and actual confidence interval coverage, compared with a parametric bootstrap, when the underlying distribution of the data is non-normal. We then apply our approach to estimate the relationship between class size and achievement for children in their reception year, after adjusting for other possible determinants. Copyright 2003 Royal Statistical Society.

Evaluation of bootstrap methods for estimating uncertainty of parameters in nonlinear mixed-effects models: a simulation study in population pharmacokinetics

“…Another issue which is very important when applying the nonparametric residual bootstrap in NLMEM is the transformation of the raw residuals to avoid the underestimation in variance parameter estimates [5,6,21]. The shrinkage correction using the ratio matrix between the empirical and the estimated variance covariance matrices was proposed by Carpenter et al, using the Cholesky decomposition for a positive definite matrix [21,22].…”

“…The shrinkage correction using the ratio matrix between the empirical and the estimated variance covariance matrices was proposed by Carpenter et al, using the Cholesky decomposition for a positive definite matrix [21,22]. This correction performed very well for LMEM [12].…”

“…That's why the nonparametric bootstrap of the raw residuals yields downwardly biased variance parameter estimates [5,6,21]. Therefore, they both must be rescaled by being centred and then transformed to have empirical variance/covariance matrices equal to those estimated by the model.…”

“…Therefore, they both must be rescaled by being centred and then transformed to have empirical variance/covariance matrices equal to those estimated by the model. In linear mixed-effects models, Carpenter et al proposed to center the random effects and residuals and then transform them using correction matrices accounting for the differences between their corresponding estimated and empirical variance-covariance matrices (shrinkage) [21,22]. The correction matrices were calculated using the Cholesky decomposition of both estimated and empirical variance-covariance matrices.…”

A comparison of bootstrap approaches for estimating uncertainty of parameters in linear mixed‐effects models

“…Previous results in the literature show that the nonparametric bootstrap of the raw residuals obtained by model fitting usually yield downwardly biased variance parameter estimates [7,8,20]. In ordinary linear models, this underestimation is due to the difference between estimated and empirical variance of residuals [21].…”