When can group level clustering be ignored? Multilevel models versus single-level models with sparse data

Abstract

Multilevel models can be reliably estimated with an average of only five observations per group. Disaggregated techniques carry an increased risk of Type I error, even in situations where there is only limited clustering in the data.

Variation in compulsory psychiatric inpatient admission in England: a cross-classified, multilevel analysis

“…It is likely, therefore, that reduced variance at patient level in models 3 and 4 is due to conflation of patient-and higher-level variance caused by clustering of patients within a small number of PCTs per Trust, a problem exacerbated by cross-classification in the data. 16, 22 We therefore considered that estimates of higher-level variance in Models 1 and 2 were more reliable than those of Models 3 and 4.…”

The Quality of Writing Tasks and Students’ Use of Academic Language in Spanish

“…Researchers employing simulation studies have found that the coefficients and standard errors produced by HLM are fairly robust to the sparse-data problem even in extreme situations (where the average number of cases at a level is two, and also even when singletons make up a fairly high proportion of cases). However, variance components at the group level may be biased upward and, likewise, the standard errors of the variance components, in particular, are likely to be biased upward, making it more difficult to detect significant group-level variation (Clarke, 2008). In light of this, we find it noteworthy that the chi-square test of significance for variation between tasks within teachers was significant for all of our academic language outcomes.…”

“…Simulation studies suggest that our fixed effects are not likely to be biased (Clarke, 2008). Furthermore, our analyses explained a fair amount of between-task and between-teacher variance even when cognitive demand and text quality were the only predictors in our models.…”

“…Recent work sheds light on the potential limitations of our findings given that our estimates were derived from a fairly small sample of tasks per teacher (see, e.g., Bell-Ellison, Ferron, & Kromrey, 2008;Clarke, 2008). Researchers employing simulation studies have found that the coefficients and standard errors produced by HLM are fairly robust to the sparse-data problem even in extreme situations (where the average number of cases at a level is two, and also even when singletons make up a fairly high proportion of cases).…”

Reasons for job separations in a cohort of workers with psychiatric disabilities

“…Absence of multicollinearity was confirmed by the fact that none of the model variables had zero-order correlations r  |0.5|. There was insufficient clustering of multiple jobs among individuals to support use of multilevel analysis [37]. Instead, data were weighted to reflect the number of jobs contributed by each individual [38].…”