The Dangers of Extreme Counterfactuals

Abstract

We address the problem that occurs when inferences about counterfactuals-predictions, ''what-if'' questions, and causal effects-are attempted far from the available data. The danger of these extreme counterfactuals is that substantive conclusions drawn from statistical models that fit the data well turn out to be based largely on speculation hidden in convenient modeling assumptions that few would be willing to defend. Yet existing statistical strategies provide few reliable means of identifying extreme counterfactuals. We offer a proof that inferences farther from the data allow more model dependence and then develop easyto-apply methods to evaluate how model dependent our answers would be to specified counterfactuals. These methods require neither sensitivity testing over specified classes of models nor evaluating any specific modeling assumptions. If an analysis fails the simple tests we offer, then we know that substantive results are sensitive to at least some modeling choices that are not based on empirical evidence. Free software that accompanies this article implements all the methods developed.

Knowledge and networks: An experimental test of how network knowledge affects coordination

“…Therefore, controlling for network features via regression adjustments is largely illusory, because we lack overlap on the relevant dimensions between the different networks (King and Zeng, 2006). In addition, if we try to include both a dummy variable for the networks and a measure of the number of connections per network the two are collinear and the regression cannot be estimated.…”

The earnout structure matters: Takeover premia and acquirer gains in earnout financed M&As

“…Such an approach provides more flexibility to address our research question compared to the strict requirement of correct specification when parametric methods and matching are used separately. Second, regarding the functional form that is adopted in the model, King and Zeng (2005) show that, by dropping observations that have no matches, the researcher not only reduces the degree of model dependence, but also needs to add less emphasis on non-linear relations and interactions among the explanatory variables. In advocating the use of parametric regressions, Ho et al (2007) do not recommend a change in the computations of the standard errors from the procedures typically used for the particular parametric method adopted.…”

Compulsory Voting, Habit Formation, and Political Participation

“…Extrapolation in this context means that the counterfactual outcome would be imputed by assigning negative weights to one or more units that did not practice compulsory voting. If this was the case, the regression approach would go beyond the support of the characteristics we actually observe for the control units which, in turn, would mean that inferences will be more model-dependent (King and Zeng 2006). This analysis requires us to derive the weights that the panel regression design implicitly assigns to the control units (Abadie, Diamond, and Hainmueller 2015).…”