An Integrated Approach to Testing Dynamic, Multilevel Theory: Using Computational Models to Connect Theory, Model, and Data

Abstract

Some of the most influential theories in organizational sciences explicitly describe a dynamic, multilevel process. Yet the inherent complexity of such theories makes them difficult to test. These theories often describe multiple subprocesses that interact reciprocally over time at different levels of analysis and over different time scales. Computational (i.e., mathematical) modeling is increasingly advocated as a method for developing and testing theories of this type. In organizational sciences, however, efforts that have been made to test models empirically are often indirect. We argue that the full potential of computational modeling as a tool for testing dynamic, multilevel theory is yet to be realized. In this article, we demonstrate an approach to testing dynamic, multilevel theory using computational modeling. The approach uses simulations to generate model predictions and Bayesian parameter estimation to fit models to empirical data and facilitate model comparisons. This approach enables a direct integration between theory, model, and data that we believe enables a more rigorous test of theory.

Multilevel Methods and Statistics: The Next Frontier

“…Ballard, Palada, Griffin, and Neal (2021) develop a technique to leverage computational and Bayesian perspectives to build and test multilevel theory in the same modeling framework. The authors’ approach involves four steps: (a) build theory into a computational model of the multilevel phenomena and use simulation to demonstrate sufficiency; (b) use empirical data to update model parameters and assess if predictions from the computation model/simulation correspond with the focal outcomes observed in the empirical setting of the study; (c) compare the results of the baseline model with alternative models representing different views about mechanisms, processes, and assumptions for the underlying theory in the baseline model; and (d) interpret model effects and assess differences between conditions or groups.…”

“…Additionally, more work is needed on empirical assessments of computational/agent-based models of emergent phenomenon and associated processes. Ballard et al (2021) provides one such approach to bring greater links between the building and assessment of these models. This approach, however, may face limitations in nonexperimental field settings, and there may be entirely different approaches that can be developed.…”

“…This is particularly important for future investigations using this approach to examine emergence in organizational contexts as such phenomena inherently involve temporal dynamics among individuals and dyads in groups (Kim et al, 2020;Liden, Anand, & Vidyarthi, 2016). Ballard et al (2021) note their framework to build and test computational multilevel models has implications for the examination of bottom-up processes at the group and higher levels of analysis (e.g., emergence or microfoundations perspectives in strategic management), and encourage future researchers to use their modeling approach to investigate such complex multilevel phenomena. The authors also call for more research on addressing computational demands and processing time for their modeling approach, as well as guidance on the setting of priors in the Bayesian aspects.…”

“…The authors’ approach involves four steps: (a) build theory into a computational model of the multilevel phenomena and use simulation to demonstrate sufficiency; (b) use empirical data to update model parameters and assess if predictions from the computation model/simulation correspond with the focal outcomes observed in the empirical setting of the study; (c) compare the results of the baseline model with alternative models representing different views about mechanisms, processes, and assumptions for the underlying theory in the baseline model; and (d) interpret model effects and assess differences between conditions or groups. While prior research has discussed ways to use computational/simulation approaches and empirical designs to investigate complex and dynamic multilevel phenomena (e.g., Grand et al, 2016; Kozlowski et al, 2013), the approach developed by Ballard and colleagues (2021) extends this prior work by providing an overarching framework that directly links the building and empirical testing of multilevel computational models.…”

“…This basic experimental design could be modified further to vary the role of the individual removed, timing of the removal, nature of the task environment, and so on. This approach would be particularly powerful if combined with the computational model building and testing technique developed by Ballard and colleagues (2021). While this suggested approach is most suited for traditional lab-oriented experiments with random assignment, it could be augmented for quasi-experiments in field settings.…”