Modeled Variance in Two-Level Models

Abstract

The concept of explained proportion of variance or modeled proportion of variance is reviewed in the situation of the random effects hierarchical two-level model. It is argued that the proportional reduction in (estimated) variance components is not an attractive parameter to represent the joint importance of the explanatory (independent) variables for modeling the dependent variable. It is preferable instead to work with the proportional reduction in mean squared prediction error for predicting individual values (for the modeled variance at level 1) and the proportional reduction in mean squared prediction error for predicting group averages (for the modeled variance at level 2). It is shown that when predictors are added, the proportion of modeled variance defined in this way cannot go down in the population if the model is correctly specified, but can go down in a sample; the latter situation then points to the possibility of misspecification. This provides a diagnostic means for identifying misspecification.

Affect labeling enhances exposure effectiveness for public speaking anxiety

“…Cohen's f 2 uses residual variance from the model to estimate effect size. However, for multi-level models, effect sizes calculated using residual variance and proportion of variance explained should be interpreted with caution because the addition of variables to the model can, in some cases, increase residual variance resulting in negative estimates of explained variance and even of effect size (Snijders & Bosker, 1994). In addition, this method cannot be used for noncontinuous dependent measures.…”

Moderating Effect of Job Type on Job Characteristic—Worker Outcome Relationships

“…τ 00 represents the variance component in the level 2 intercept term and if significant, this indicates some jobs have higher mean scores on a given worker outcome than others. Furthermore, τ 11 represents the variance component in the level 2 slope term and if significant, it indicates that there is variability in the level 1 slopes that is unexplained by the level 2 predictors (Snijders & Bosker, 1994). Therefore, all results tables for multilevel analyses conducted include variance components and indicate whether they are significant.…”

“…Effect sizes for level 1 as well as for all level 2 relationships were computed using Snijders and Bosker's (1994) technique of computing total variance explained which utilizes both level 1 and 2 variance components to avoid issues of negative values that commonly occur when computing amount of variance explained in a multilevel model using other effect size calculations. In their empirical test of various measures for computing variance explained in multilevel models, LaHuis, Hartman, Hakoyama, and Clark (2014) found the S&B technique to be an appropriate manner of calculating explained variance.…”

“…In their empirical test of various measures for computing variance explained in multilevel models, LaHuis, Hartman, Hakoyama, and Clark (2014) found the S&B technique to be an appropriate manner of calculating explained variance. Equation 4 shows Snijders and Bosker's (1994) technique. All multilevel analyses results tables include these effect sizes.…”

A Life Course Perspective on Health Trajectories and Transitions

“…In other words, the unexplained variance of the models should decrease as we add important predictors (this effect can be tested statistically). Effect size measures, similar to the coefficient of determination (R 2 ) in ordinary linear regression, have been proposed to quantify these effects (e.g., Snijders and Bosker 1994).…”