Back to Basics with Mixed-Effects Models: Nine Take-Away Points

Abstract

Purpose Multilevel mixed effects models are widely used in organizational behavior and organizational psychology to test and advance theory. At times, however, the complexity of the models leads researchers to draw erroneous inferences or otherwise use the models in less than optimal ways. We present nine take-away points intended to enhance the theoretical precision and utility of the models. Approach We demonstrate our points using two types of simulated data: one in which group membership is irrelevant, and the other in which relationships exist only because of group membership. We then demonstrate that the effects we observe in simulated data replicate in organizational data. Findings Little that we address will be new to methodology experts; nonetheless, we draw together a variety of points that we believe will help advance both theory and analytic rigor in multilevel analyses. Implications We make two points that run somewhat counter to conventional norms. First, we argue that mixed-effects models are appropriate even when ICC(1) values associated with the outcome data are small and non-significant. Second, we show that high ICC(2) values are not a prerequisite for detecting emergent multilevel relationships. Originality/ValueThe article is designed to be a resource for researchers who are learning about and applying mixed-effects (i.e., multilevel) models. are routinely used to analyze data in organizational behavior and organizational psychology. Researchers recognize that nested data are often non-independent such that responses on the dependent variable from members of the same group are more similar than would be expected by chance (Bliese 2000). The idea that mixed-effects models can account for non-independence is well documented; however, options surrounding model specification and interpreting parameter estimates from mixed-effects models are not always straightforward. As such, researchers occasionally underutilize mixedeffects models, engage in inappropriate model building, or misinterpret model parameters.Our goals are to (a) clarify situations where mixed-effects models may be useful and (b) explain the interpretation of results in common variants of the mixed-effects model. Much of what we cover will be known to methodologists; nonetheless, we believe that going back to basic ideas can help researchers more effectively use these methods to test and advance theory. We also emphasize that two of the points we raise (using mixed-effects models when levels of nonindependence are minimal and detecting emergent effects when group-mean reliability is low) run counter to conventional norms, so we encourage authors, editors, and reviewers to reconsider these two points in particular.

The consequences of (not) seeing eye‐to‐eye about the past: The role of supervisor–team fit in past temporal focus for supervisors' leadership behavior

“…Finally, the ICC(2) values for task‐oriented and relationship‐oriented leadership indicated acceptable reliability, although we note that the respective value was lower than desirable for laissez‐faire leadership. In sum, this overall pattern of aggregation statistics suggests that it was justified to aggregate individual members' assessments of their supervisors' leadership behavior to the team level of analysis (Bliese, Maltarich, & Hendricks, ).…”

A Multilevel Person-Centered Examination of Teachers’ Workplace Demands and Resources: Links With Work-Related Well-Being

“…For example, leadership style and schoolaverage teacher collaboration have been associated with schoolaverage teachers' levels of organizational commitment (Duyar et al, 2013) and self-efficacy (Fackler and Malmberg, 2016). Although the variance explained by school-level factors can often be relatively modest, the school-level is nonetheless important to consider from a measurement perspective (Bliese et al, 2018) and given prior work demonstrating that school-level phenomena have a role to play in teachers' outcomes (e.g., Duyar et al, 2013). Moving forward, it is important to extend this variablecentered knowledge to a person-centered understanding.…”

Team Focus in Focus: Its Implications for Real Teams and Their Members

“…We found the ICC(1) value for team focus in our sample to be .01. Although this ICC(1) value is relatively small, ICC(1) values larger than zero indicate a multilevel model will produce more accurate results than a single-level model (Bliese, Maltarich, & Hendricks, 2018;Tofighi & Thoemmes, 2014). As further support for aggregation, we found the mean r wg(j) for team focus was .87.…”