Construct Meaning in Multilevel Settings

Abstract

We present types of constructs, individual- and cluster-level, and their confirmatory factor analytic validation models when data are from individuals nested within clusters. When a construct is theoretically individual level, spurious construct-irrelevant dependency in the data may appear to signal cluster-level dependency; in such cases, however, and consistent with theory, a single-level analysis with a correction for dependency may be appropriate. Regarding cluster-level constructs, we discuss two types—shared and configural—and present appropriate validation models. Illustrative validation analyses with individual, shared, and configural constructs are provided using empirical data as well as simple simulations demonstrating the spurious effects that can occur with nested data. The article concludes with future directions to be examined in construct validation in multilevel settings.

The Role of Classroom Discussion

“…Lüdtke et al (2009) argued that the study of school environments should center its attention on the between-school differences when students are the informants. This translates into appropriately identifying if a measure is a reflective construct of a cluster level (Stapleton et al 2016), and using appropriate centering techniques for responses. In practice, this treats student answers as if they are raters of their own learning environment.…”

“…Open classroom discussion is a reflective measure of the school environment (Lüdtke et al 2008;Stapleton et al 2016) and not a classical individual difference measure. Its frame of reference is the learning environment and not just the experience of students as individuals.…”

Avoiding bias and fallacy in survey research: A behavioral multilevel approach

“…The only actual measures in this model reside at the within level. As Stapleton, Yang, and Hancock (, p. 492, emphasis added) noted, when item responses come from individuals, “[t]he nature of the latent construct at the cluster level can be perplexing .” The between factors can indeed appear somewhat perplexing, but they are presented here as plausible interpretations of the between factor structure.…”

“…In contrast with the theory‐based within model of commitment, the aim at the between level in this example is simply to specify a between factor structure that is substantively more plausible than a model that simply assumes the between model to have the same structure as the within model. As Muthén (, p. 388) noted, “[t]he between components have a different meaning than the within components, and it is not clear that the between‐group covariation follows a simple model.” The issue is fundamentally not about statistics, it is about the meaning of the concepts (see also Stapleton et al, ). The concern is also theoretical, because the within and between parts of the MFA model “tap into constructs that function at different levels and […] should always be labeled differently, with the content of the labels appropriate for the level of analysis to which they are applied” (Bond, , p. 567).…”

Teaching Tolerance in a Globalized World: Final Remarks

“…Future research could address the need to better differentiate both theoretically and empirically between school and class-related factors and their relationship to young peoples' egalitarian attitudes (using sampling strategies targeting more than one class per school). Complex methodological and theoretical questions pose further aspects for reflection, including identifying the most appropriate measurement models for students' ratings of classrooms, addressing plausible endogeneity in modelling multilevel data, and unfolding the complex explanatory mechanisms (the why, how, and when) that link classroom factors to student attitudes (see Stapleton et al 2016).…”