Detecting Consensus Emergence in Organizational Multilevel Data: Power Simulations

Abstract

Theories suggest that groups within organizations often develop shared values, beliefs, affect, behaviors, or agreed-on routines; however, researchers rarely study predictors of consensus emergence over time. Recently, a multilevel-methods approach for detecting and studying emergence in organizational field data has been described. This approach—the consensus emergence model—builds on an extended three-level multilevel model. Researchers planning future studies based on the consensus emergence model need to consider (a) sample size characteristics required to detect emergence effects with satisfactory statistical power and (b) how the distribution of the overall sample size across the levels of the multilevel model influences power. We systematically address both issues by conducting a power simulation for detecting main and moderating effects involving consensus emergence under a variety of typical research scenarios and provide an R-based tool that readers can use to estimate power. Our discussion focuses on the future use and development of multilevel methods for studying emergence in organizational research.

Specifying Location–Scale Models for Heterogeneous Variances as Multilevel SEMs

“…Results show only slight differences due to different estimation methods. Table 3 Comparison of estimates from the Model 3c in Lang et al (2019), the reproduction of that model as a Bayesian multilevel SEM, and the extension of the model that allows for scale random effects estimated as a multilevel SEM. Note: Model 3c = the model originally presented in Table 9 of Lang et al (2018).…”

“…Despite the frequency of questions about cohesion and heterogeneity in the organizational literature, Lang et al (2018Lang et al ( , 2019 for a heterogeneous variance model in a mixed effect framework and would likely require a multistage model whose estimates would require Croon's correction (Croon, 2002) in order to be unbiased (Devlieger & Rosseel, 2017;Kelcey, Cox, & Dong, 2019), mediation is another straightforward extension if modeling the random scale effects in the multivariate multilevel SEM framework (Hoffman, 2019). McNeish and Hamaker (2020) note such advantages of multilevel SEM over mixed effects models, noting that multilevel SEM allows any hypothesized structural relations to be modeled between random effects.…”

“…The full code is included on the Open Science Framework Page and results are shown in the last column ofTable 3. This would WhereasLang et al (2019) note the hurdles for fitting this type of model in the mixed effect framework with frequentist estimation, allowing for all of these aspects presents no issues in the Bayesian multilevel SEM framework, even with only 34 units. In fact, the model converged in only 31 seconds, vastly eclipsing the demanding nonlinear estimation that would be required if trying to fit the model in another suitable software environment such as SAS Proc Nlmixed.…”

“…However, these models can be further expanded with the inclusion of random scale effects to quantify unexplained variance in the scale submodel and protect against inflated Type-I error rates for fixed effects that can arise from misspecifying the random effect structure.Adopting location-scale models in a multilevel SEM framework would seem to have particular advantages for addressing the possible limitations noted with applications of heterogeneous variance models in the recent literature. For instance,Lang et al (2019) note random scale effects as a future direction with the qualification that such models often cannot include random location slopes on account of their computational encumbrances (e.g., the MIXREGLS program does not permit random location slopes). As we show in our examples, location-scale models in the multilevel SEM framework can accommodate multiple scale submodel random effects and can correlate them with location scale effects without issue.…”

“…As we show in our examples, location-scale models in the multilevel SEM framework can accommodate multiple scale submodel random effects and can correlate them with location scale effects without issue. Furthermore, the computational burden was essentially non-existent as the models converged in less than one minute Lang et al (2019). also discuss the possibility of merging heterogeneous variance models with other common organizational techniques like mediation models but note "it is not clear how the approach [mediation] could be extended to emergence models" (p. 15).…”