How Computational Modeling Can Force Theory Building in Psychological Science

Abstract

Psychology endeavors to develop theories of human capacities and behaviors on the basis of a variety of methodologies and dependent measures. We argue that one of the most divisive factors in psychological science is whether researchers choose to use computational modeling of theories (over and above data) during the scientific-inference process. Modeling is undervalued yet holds promise for advancing psychological science. The inherent demands of computational modeling guide us toward better science by forcing us to conceptually analyze, specify, and formalize intuitions that otherwise remain unexamined—what we dub open theory. Constraining our inference process through modeling enables us to build explanatory and predictive theories. Here, we present scientific inference in psychology as a path function in which each step shapes the next. Computational modeling can constrain these steps, thus advancing scientific inference over and above the stewardship of experimental practice (e.g., preregistration). If psychology continues to eschew computational modeling, we predict more replicability crises and persistent failure at coherent theory building. This is because without formal modeling we lack open and transparent theorizing. We also explain how to formalize, specify, and implement a computational model, emphasizing that the advantages of modeling can be achieved by anyone with benefit to all.

Replication is important for educational psychology: Recent developments and key issues

“…Schmidt ( 2017) is more direct, noting that, " … a single observation cannot be trusted," and that "replication … is capable of transforming an observation into a fact, or piece of knowledge" (p. 236, emphasis in original). Several scholars have recently argued replication plays an important role in theory building and theory assessment (Guest & Martin, 2021;Irvine, 2021;van Rooij & Baggio, 2021).…”

“…Viewed conceptually, a replication adds value if it helps build theory (Guest & Martin, 2021), narrow or assess a given theory (Irvine, 2021), or assess whether an explanation has particular boundaries on effects (van Rooij & Baggio, 2021). In particular, high rates of replication may be needed in studies based on weak theory, a lot of variability in results across contexts, or theory that predicts many complex factors affecting outcomes.…”

A practical guide for studying human behavior in the lab

“…In summary, pre-registrations (Lakens, 2019) and replications (R A Klein et al, 2018;Richard A. Klein et al, 2014) will help to improve the standards of science, partially by protecting against involuntary malpractices, and will greatly strengthen your results in the eyes of your reviewers and readers. Beware however that preregistration and replication cannot replace a solid theoretical embedding of your hypothesis (Guest & Martin, 2021;Szollosi et al, 2020) .…”

“…Broadly, statistical and computational modeling can buy you four things: i) Model fitting, to quantitatively estimate relevant effects and compare them between different conditions or populations, ii) Model validation , to test if your conceptual model captures how behavior depends on the experimental variables, iii) Model comparison, to determine quantitatively which of your hypothesized models is best supported by your data and iv) Model predictions that are derived from your data and can be tested in new experiments. On a more general level, computational modeling can constrain the space of possible interpretations of your data, and therefore contributes to reproducible science and more solid theories of the mind (Guest & Martin, 2021) .…”

Replication is Important for Educational Psychology: Recent Developments and Key Issues

“…Theory matters (Gehlbach & Robinson, 2021/this issue;Smith et al, 2017;Wentzel, 2021/this issue), and having carefully constructed theories that build on theories and research of the past serve as the foundation for all we do as applied psychologists (Vartanian, 2017). The growing discussion of replication's role in theory development and assessment (e.g., Guest & Martin, 2021;Irvine, 2021;van Rooij & Baggio, 2021) will likely have a major impact on replication use. For example, making decisions about when replications are needed and what types of replication should be used may be informed by what they contribute to the development of a particular theory.…”