When Good Balance Goes Bad: A Discussion of Common Pitfalls When Using Entropy Balancing

Abstract

For many accounting research questions, empirical researchers cannot randomly assign observations to treatment conditions or identify a quasi-experimental setting. In these cases, entropy balancing (Hainmueller 2012) is an increasingly popular statistical method for identifying a control sample that is nearly identical to the treated sample with respect to observable covariates. In this paper, we compare entropy balancing’s approach of reweighting control sample observations to ordinary least squares and propensity score matching. We demonstrate that researchers applying entropy balancing in empirical settings involving panel data with features common in accounting research may encounter implementation issues that render the resulting estimates sensitive to relatively minor changes in the control sample or the research design. Using the setting of estimating the Big-N audit fee premium, we empirically demonstrate these issues and propose solutions.

Auditors’ Use of In‐House Specialists

“…Section A3 provides descriptive statistics about the entropy balancing procedures used to estimate model 1 in table 5. By construction(McMullin and Schonberger [2020, 2022]), applying the balancing weights is effective at mitigating differences in client and auditor characteristics (i.e., standardized differences in means are zero for all variables after applying the weights). Section A4 reports sensitivity tests following the approach suggested by Altonji, Elder, and Taber [2005], indicating that our findings are unlikely to be driven by selection issues.…”

“…All the control variables are used to determine the entropy balancing weights. This approach helps mitigate concerns that engagements with low or high specialist involvement differ on observable characteristics(McMullin and Schonberger [2020, 2022]). The partition of specialist use at the median is relatively straightforward.…”

Financial statement comparability and accounting fraud

“…Finally, McMullin and Schonberger (2022) note that while entropy balancing has many advantages as a matching procedure, they highlight some key challenges of implementing this technique when using panel data, especially in situations where the assignment to treatment and control groups is not clear (such as in our setting, where the assignment is based on comparability). McMullin and Schonberger (2022) identify two main issues of entropy balancing in these settings—the assignment of extreme weights to some control observations and covariate balancing across years in a pooled sample.…”

“…Finally, McMullin and Schonberger (2022) note that while entropy balancing has many advantages as a matching procedure, they highlight some key challenges of implementing this technique when using panel data, especially in situations where the assignment to treatment and control groups is not clear (such as in our setting, where the assignment is based on comparability). McMullin and Schonberger (2022) identify two main issues of entropy balancing in these settings—the assignment of extreme weights to some control observations and covariate balancing across years in a pooled sample. The assignment of extreme weights to control sample observations (likely to occur when there are systematic differences in treatment and control covariates) raises concerns about how reproducible the estimate of the treatment effect is with alternate samples.…”

“…) McMullin and Schonberger (2022). identify two main issues of entropy balancing in TA B L E 5 Financial statement comparability and accounting fraud i 𝛾 i Firm i + ∑ t 𝛿 t Year t + 𝜀 it .…”