Modelling Method Effects as Individual Causal Effects

Abstract

Method effects often occur when different methods are used for measuring the same construct. We present a new approach for modelling this kind of phenomenon, consisting of a definition of method effects and a first model, the "method effect model", that can be used for data analysis. This model may be applied to multitrait-multimethod data or to longitudinal data where the same construct is measured with at least two methods at all occasions. In this new approach, the definition of the method effects is based on the theory of individual causal effects by Neyman and Rubin. Method effects are accordingly conceptualized as the individual effects of applying measurement method "j" instead of "k". They are modelled as latent difference scores in structural equation models. A reference method needs to be chosen against which all other methods are compared. The model fit is invariant to the choice of the reference method. The model allows the estimation of the average of the individual method effects, their variance, their correlation with the traits (and other latent variables) and the correlation of different method effects among each other. Furthermore, since the definition of the method effects is in line with the theory of causality, the method effects may (under certain conditions) be interpreted as causal effects of the method. The method effect model is compared with traditional multitrait-multimethod models. An example illustrates the application of the model to longitudinal data analysing the effect of negatively (such as 'feel bad') as compared with positively formulated items (such as 'feel good') measuring mood states. Copyright 2008 Royal Statistical Society.

Further Validation and Development of the Movement Imagery Questionnaire

“…Similarly to Study 3, it would be worthwhile to investigate the predictive nature of the MIQ-3 compared with the MIQ-R for outcomes such as performance and psychological characteristics that influence performance, such as anxiety and confidence. Although the current study tested the four MTMM models advocated by Marsh (1989), future research could investigate a potential multiplicative structure by testing a direct product model and individual level method effects using the more recent MTMM approach proposed by Pohl, Steyer and Kraus (2008) to examine the model fit of the MIQ-R and MIQ-3. Research could also use nonrecursive and longitudinal models to investigate if there is a more substantive element to the cross-trait association between the same items such as a relationship between the modalities.…”

Multitrait-multimethod change modelling

“…Finally, it should be mentioned that there are other approaches to MTMM analysis that do not define method effects in terms of residual scores (e.g., Pohl and Steyer 2008). Depending on the research questions these approaches may be more suitable in some cases (see Eid et al 2008 for an overview of different approaches).…”

Modeling Common Traits and Method Effects in Multitrait-Multimethod Analysis

“…A new model in which the method factors are not modeled as residuals but as "effects" is the Method Effect model with a reference method (MEref; Pohl et al, 2008). In this model, a reference method needs to be chosen and the trait factors (£ t1 ) are, as in the CTC(M-1) model, defined as the true-score variable of the manifest variable measuring the respective trait with the reference method.…”

“…The most frequently applied models are the correlated trait-correlated uniqueness model (CTCU; Kenny, 1976;Marsh, 1989;Marsh & Craven, 1991), the correlated trait-correlated methods model (CTCM; Jöreskog, 1974;Widaman, 1985), and the correlated trait-correlated method minus one model (CTC [M-1]; Eid, 2000). A more recent model is the Method Effect model with a reference method (Pohl, Steyer, & Kraus, 2008). These models are depicted in Figure 1 for nine manifest variables Y tj measuring three traits (t) by three methods (j ).…”