Approximate Measurement Invariance of Willingness to Sacrifice for the Environment Across 30 Countries: The Importance of Prior Distributions and Their Visualization

Abstract

Nationwide opinions and international attitudes toward climate and environmental change are receiving increasing attention in both scientific and political communities. An often used way to measure these attitudes is by large-scale social surveys. However, the assumption for a valid country comparison, measurement invariance, is often not met, especially when a large number of countries are being compared. This makes a ranking of countries by the mean of a latent variable potentially unstable, and may lead to untrustworthy conclusions. Recently, more liberal approaches to assessing measurement invariance have been proposed, such as the alignment method in combination with Bayesian approximate measurement invariance. However, the effect of prior variances on the assessment procedure and substantive conclusions is often not well understood. In this article, we tested for measurement invariance of the latent variable “willingness to sacrifice for the environment” using Maximum Likelihood Multigroup Confirmatory Factor Analysis and Bayesian approximate measurement invariance, both with and without alignment optimization. For the Bayesian models, we used multiple priors to assess the impact on the rank order stability of countries. The results are visualized in such a way that the effect of different prior variances and models on group means and rankings becomes clear. We show that even when models appear to be a good fit to the data, there might still be an unwanted impact on the rank ordering of countries. From the results, we can conclude that people in Switzerland and South Korea are most motivated to sacrifice for the environment, while people in Latvia are less motivated to sacrifice for the environment.

Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives

“…Therefore, it is essential to select the hyperparameters carefully (i.e., values for σ 0 of λ pg or ν pg for all group differences) and provide a rationale for the choice of the hyperparameters. Arts et al (2021) provide an overview of studies applying BAMI with hyperparameter specifications and the underlying rationale of this choice-if present at all; see their supplementary materials at the Open Science Framework: https://osf.io/t3h9e. First of all, users of BAMI and any other Bayesian method should always include a rationale of their prior specification; see also the Bayesian analysis reporting guidelines by Kruschke (2021) or the "When to Worry and How to Avoid the Misuse of Bayesian Statistics" checklist by Depaoli and van de Schoot (2017).…”

Estimation Methods of the Multiple-Group One-Dimensional Factor Model: Implied Identification Constraints in the Violation of Measurement Invariance

“…With equal weights w 1ig for all items within a group g, this shows that BAMI and ML pose the same identification constraints on ν * ig ; that is, ∑ I i=1 ν * ig = 0. A variant of IA has been proposed that uses the output of BAMI for determining the alignment solution [45][46][47]. BAMI produces adjusted group-specific item means µ ig = α g + ν i0 + ν ig , which are subsequently used as input data for Bayesian IA.…”

“…It can be seen that MNI in loadings due to λ * ig as well as MNI in intercepts due to ν * ig determine the group mean α g . For additive deviations from MI that follow (45), the condition ( 46) is replaced by…”

“…We now discuss identification constraints in regularized ML estimation of the multiplegroup tau-congeneric measurement model. Deviations of MI in item loadings can be either modeled in a multiplicative (see (44)) or an additive (see (45)) form. In the following, we assume additive effects because this specification appears more frequently in practical applications.…”

“…As also discussed in Section 3.3 for the tau-equivalent model, the output of BAMI can be used in a subsequent IA estimation [45]. Using the same steps as in Section 3.3 for the derivations, an identification constraint similar to (36) can be obtained.…”