A longitudinal multilevel CFA-MTMM model for interchangeable and structurally different methods

Abstract

One of the key interests in the social sciences is the investigation of change and stability of a given attribute. Although numerous models have been proposed in the past for analyzing longitudinal data including multilevel and/or latent variable modeling approaches, only few modeling approaches have been developed for studying the construct validity in longitudinal multitrait-multimethod (MTMM) measurement designs. The aim of the present study was to extend the spectrum of current longitudinal modeling approaches for MTMM analysis. Specifically, a new longitudinal multilevel CFA-MTMM model for measurement designs with structurally different and interchangeable methods (called Latent-State-Combination-Of-Methods model, LS-COM) is presented. Interchangeable methods are methods that are randomly sampled from a set of equivalent methods (e.g., multiple student ratings for teaching quality), whereas structurally different methods are methods that cannot be easily replaced by one another (e.g., teacher, self-ratings, principle ratings). Results of a simulation study indicate that the parameters and standard errors in the LS-COM model are well recovered even in conditions with only five observations per estimated model parameter. The advantages and limitations of the LS-COM model relative to other longitudinal MTMM modeling approaches are discussed.

Different kinds of interchangeable methods in multitrait-multimethod analysis: a note on the multilevel CFA-MTMM model by Koch et al. (2014)

“…While the former correspond with pure hierarchical data structures and meet the assumptions of the existing multilevel CFA-MTMM models proposed by Koch et al (2014) and Eid et al (2008), the latter correspond with crossed hierarchical data and violate the assumptions. Equation (5) shows how multilevel CFA-MTMM models can be modified to account for crossed random effects of targets and raters that arise from interchangeable methods with overlapping samples of raters.…”

“…Assuming that targets are rated on multiple indicators i , this problem has been resolved by the specification of a unidimensional measurement model for the rater effects UM trijkl across indicators, $U{M}_{trijkl}={\lambda }_{U{M}_{ijkl}}U{M}_{trjkl}$ (see Eid et al, 2008, p. 233; Koch et al, 2014, p. 7). In Equation (3), UM trjkl reflects the random effect of rater r nested under target t which is considered constant across indicators i , and λ UM ijkl is the respective factor loading of indicator i 1 .…”

“…It remains a challenge for future research to implement the cross-classified target and rater effects S tijkl and R rijkl in practical data analysis tools for parameter estimation and model testing of multilevel CFA-MTMM models that extend the tools introduced by Eid et al (2008) and Koch et al (2014). …”

Structural equation modeling of multiple-indicator multimethod-multioccasion data: A primer

“…Models for more complex MM designs. Koch, Schultze, Eid, and Geiser (2014) presented an extended CS-C(M -1) model for MM measurement designs that use both structurally different (fixed) and interchangeable (random) methods. A similar extension of Courvoisier et al's (2008) MM-LST model is discussed in Koch et al (in press).…”

Analysing multisource feedback with multilevel structural equation models: Pitfalls and recommendations from a simulation study

“…There are many simulation studies and much analytic research on the performance of multilevel models. They have demonstrated that the required sample sizes strongly depend on model characteristics, mainly its complexity (Koch, Schultze, Eid, & Geiser, ; Meuleman & Billiet, ). Therefore, one cannot directly derive recommendations for the application of the model by Mahlke et al .…”