Purpose Multilevel mixed effects models are widely used in organizational behavior and organizational psychology to test and advance theory. At times, however, the complexity of the models leads researchers to draw erroneous inferences or otherwise use the models in less than optimal ways. We present nine take-away points intended to enhance the theoretical precision and utility of the models. Approach We demonstrate our points using two types of simulated data: one in which group membership is irrelevant, and the other in which relationships exist only because of group membership. We then demonstrate that the effects we observe in simulated data replicate in organizational data. Findings Little that we address will be new to methodology experts; nonetheless, we draw together a variety of points that we believe will help advance both theory and analytic rigor in multilevel analyses. Implications We make two points that run somewhat counter to conventional norms. First, we argue that mixed-effects models are appropriate even when ICC(1) values associated with the outcome data are small and non-significant. Second, we show that high ICC(2) values are not a prerequisite for detecting emergent multilevel relationships.
Originality/ValueThe article is designed to be a resource for researchers who are learning about and applying mixed-effects (i.e., multilevel) models. are routinely used to analyze data in organizational behavior and organizational psychology. Researchers recognize that nested data are often non-independent such that responses on the dependent variable from members of the same group are more similar than would be expected by chance (Bliese 2000). The idea that mixed-effects models can account for non-independence is well documented; however, options surrounding model specification and interpreting parameter estimates from mixed-effects models are not always straightforward. As such, researchers occasionally underutilize mixedeffects models, engage in inappropriate model building, or misinterpret model parameters.Our goals are to (a) clarify situations where mixed-effects models may be useful and (b) explain the interpretation of results in common variants of the mixed-effects model. Much of what we cover will be known to methodologists; nonetheless, we believe that going back to basic ideas can help researchers more effectively use these methods to test and advance theory. We also emphasize that two of the points we raise (using mixed-effects models when levels of nonindependence are minimal and detecting emergent effects when group-mean reliability is low) run counter to conventional norms, so we encourage authors, editors, and reviewers to reconsider these two points in particular.