Correcting Measures of Relationship between Aggregate-Level Variables

O’Brien, Robert M.

doi:10.2307/270934

Cited by 4 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the last term of students within schools effects is compounded by unexplained sources of error (O'Brien 1991). Suppose that schools are the objects of measurement, in this case, the universe of generalization consist of a random p facet.…”

Section: School-level Score Estimation Methodsmentioning

confidence: 99%

“…Generalizability theory has been commonly used for this purpose (Brennan 1995;Gao et al 1994;Jones and Norrander 1996;O'Brien 1991), as it enables investigators to explore reliabilities for various circumstances by fixing or randomizing measurement conditions (Brennan 2001a, b). Though multilevel models have not been frequently used for this purpose, they do offer many advantages in examining the relationships among individual-level and school-level measures (Raudenbush and Bryk 1986;Teddlie and Reynolds 2000).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Estimating reliability of school-level scores using multilevel and generalizability theory models

Jeon

Lee

Hwang

et al. 2009

Asia Pacific Educ. Rev.

View full text Add to dashboard Cite

The purpose of this study was to investigate the methods of estimating the reliability of school-level scores using generalizability theory and multilevel models. Two approaches, 'student within schools' and 'students within schools and subject areas,' were conceptualized and implemented in this study. Four methods resulting from the combination of these two approaches with generalizability theory and multilevel models were compared for both balanced and unbalanced data. The generalizability theory and multilevel models for the 'students within schools' approach produced the same variance components and reliability estimates for the balanced data, while failing to do so for the unbalanced data. The different results from the two models can be explained by the fact that they administer different procedures in estimating the variance components used, in turn, to estimate reliability. Among the estimation methods investigated in this study, the generalizability theory model with the 'students nested within schools crossed with subject areas' design produced the lowest reliability estimates. Fully nested designs such as (students:schools) or (subject areas:students:schools) would not have any significant impact on reliability estimates of school-level scores. Both methods provide very similar reliability estimates of school-level scores.

show abstract

Section: School-level Score Estimation Methodsmentioning

confidence: 99%