Field experiments in operations management

Abstract

While the field experiment is a powerful and well‐established method to investigate causal relationships, operations management (OM) has embraced this methodology only in recent years. This paper provides a comprehensive review of the existing OM literature leveraging field experiments. It also serves as a one‐stop guide for future application of field experiments in the OM area. We start by recapping the characteristics that distinguish field experiments from other common types of experiments and organizing the relevant OM studies by topic. Corresponding to the commonly overlooked issues in field experiment‐based OM studies, we then provide a detailed roadmap, ranging from experimental design and implementation to post‐experiment analysis. We further outline the methodological and practical issues as well as corresponding solutions when applying field experiments. We conclude by identifying future research directions from an OM perspective.

Emphasizing worker identification with skills to increase helping and productivity in production: A field experiment

“…Generally, the interest in publishing null results is increasing (Mervis, 2014; Miller & Bamberger, 2016), and this special issue review process has significantly decreased possible publication biases. As the interest in field experiments of operations management scholars is increasing (Eckerd et al, 2020; Gao et al, 2023), innovative pre‐registration publication processes are one way to ensure that studies using field experiments will not choose increasingly incremental and low‐risk research questions to avoid the publication bias. Of course, null results are of significant managerial importance as they point to avenues for managerial intervention that are potentially less productive or point to the need to carefully adapt treatments before applying them again in the field or laboratory.…”

Causal inference under selection on observables in operations management research: Matching methods and synthetic controls

“…To provide a practical guideline, in line with recent methodological reviews in JOM (Chou et al, 2023; Gao et al, 2023; Lu et al, 2018), we present a flow diagram illustrating the main steps for implementing the PSM method (Figure 2). These steps will be further discussed in Sections 3.1–3.4.…”

“…While a randomized experiment is the most desirable setting for drawing causal inferences on the effects of a treatment, conducting such experiments can be difficult, if not impossible, for certain research questions. Lonati et al (2018) and Eckerd et al (2021) discuss challenges for lab experiments, while Gao et al (2023) cover challenges for field experiments. As a result, the majority of OM studies investigating the effect of a treatment rely on existing data, either collected by the researchers from the focal organization or a third party, or obtained through techniques such as web crawling.…”

Registered reports in operations management: Lessons from an experimental trial

“…Moreover, there are excellent resources to support the development of capabilities in this area. For example, the field of economics offers guides on field experiments (e.g., Levit & List, 2009; List, 2011) and both Ibañez and Staats (2019) and Gao and Li (2023) provide perspectives grounded in the field of operations. Evaluating papers appropriately means finding the right mix of reviewers with respect to methods and topics.…”