We evaluate whether a lack of guidance on how to choose the matching variables used in the Synthetic Control (SC) estimator creates specification-searching opportunities. We provide theoretical results showing that specification-searching opportunities are asymptotically irrelevant if we restrict to a subset of SC specifications. However, based on Monte Carlo simulations and simulations with real datasets, we show significant room for specification searching when the number of pre-treatment periods is in line with common SC applications, and when alternative specifications commonly used in SC applications are also considered. This suggests that such lack of guidance generates a substantial level of discretion in the choice of the comparison units in SC applications, undermining one of the advantages of the method. We provide recommendations to limit the possibilities for specification searching in the SC method. Finally, we analyze the possibilities for specification searching and provide our recommendations in a series of empirical applications. C 2020 by the Association for Public Policy Analysis and Management on behalf of the Association for Public Policy Analysis and Management variables, the choice of how to split the pre-treatment periods into training and validation periods, and even the choice of software and data-sorting criteria (see Klößner et al., 2017, for details on this last point). Therefore, our results should be seen as a lower bound on the possibilities for specification searching in SC applications. We focus on the choice of pre-treatment outcome lags, rather than on the inclusion of covariates, because it is possible to systematically analyze the inclusion of pre-intervention outcomes lags in a way that encompasses all applications, while covariates may differ in complex ways from one application to another. We consider the possibility of specification searching in the decision to include covariates in our empirical applications. 6 Appendices are available at the end of this article as it appears in JPAM online. Go to the publisher's website and use the search engine to locate the article at http://onlinelibrary.wiley.com. 7 In Appendix B, we also consider specifications that use time-invariant covariates as predictors, in addition to functions of the pre-treatment outcomes. All results remain similar. This package solves the nested minimization problem described by equations (1) and (2). We specify the optimization method to be BFGS only and use optimization routine Low Rank Quadratic Programming when Interior Point optimization routine does not converge. 24 When we adopt this decision rule in our MC simulations, then the probability of rejecting the null at 5 percent for all specifications is lower than 1 percent in all scenarios. If we discard specifications 6 and 7, then this rejection rate ranges from 1 percent when T 0 = 12 to 2.8 percent when T 0 = 400. 25 Following the best practices in terms of transparency and replicability, Hainmueller (2014) made their dataset and replication ...

Differences-in-Differences (DID) is one of the most widely used identification strategies in applied economics. However, inference in DID models when there are few treated groups is still an open question. We show that usual inference methods used in DID models might not perform well when there are few treated groups and residuals are heteroskedastic. In particular, when there is variation in the number of observations per group, we show that inference methods designed to work when there are few treated groups would tend to (under-) over-reject the null hypothesis when the treated groups are (large) small relative to the control groups. This happens because larger groups would have lower variance, generating heteroskedasticity in the group x time aggregate DID model. We provide evidence from Monte Carlo simulations and from placebo DID regressions with the American Community Survey (ACS) dataset to show that this problem is relevant even in datasets with large number of observations per group. Then we derive alternative inference methods that provide accurate hypothesis testing in situations of few treated groups and many control groups in the presence of heteroskedasticity (including the case of only one treated group). The main assumption is that we know how the heteroskedasticity is generated, which is the case when it is generated by variation in the number of observations per group. Finally, we also show that an inference method for the Synthetic Control Estimator proposed by Abadie et al. (2010) can correct for the heteroskedasticity problem, and derive conditions under which this inference method provides accurate hypothesis testing.

We derive conditions under which the original result of Abadie et al. (2010) regarding the bias of the Synthetic Control (SC) estimator remains valid when we relax the assumption of a perfect match on observed covariates. We then show that, even when the conditions for the first result are valid, a perfect match on pre-treatment outcomes does not generally imply an approximate match for all covariates. Taken together, our results show that a perfect match on covariates should not be required for the SC method, as long as there is a perfect match on a long set of pre-treatment outcomes.JEL codes: C13, C21, C23.

We analyze the properties of the Synthetic Control (SC) and related estimators when the pre‐treatment fit is imperfect. In this framework, we show that these estimators are generally biased if treatment assignment is correlated with unobserved confounders, even when the number of pre‐treatment periods goes to infinity. Still, we show that a demeaned version of the SC method can improve in terms of bias and variance relative to the difference‐in‐difference estimator. We also derive a specification test for the demeaned SC estimator in this setting with imperfect pre‐treatment fit. Given our theoretical results, we provide practical guidance for applied researchers on how to justify the use of such estimators in empirical applications.

Using a high-stakes field experiment conducted with a financial brokerage, we implement a novel design to separately identify two channels of social influence in financial decisions, both widely studied theoretically. When someone purchases an asset, his peers may also want to purchase it, both because they learn from his choice ("social learning") and because his possession of the asset directly affects others' utility of owning the same asset ("social utility"). We randomize whether one member of a peer pair who chose to purchase an asset has that choice implemented, thus randomizing his ability to possess the asset. Then, we randomize whether the second member of the pair: (i) receives no information about the first member, or (ii) is informed of the first member's desire to purchase the asset and the result of the randomization that determined possession. This allows us to estimate the effects of learning plus possession, and learning alone, relative to a (no information) control group. We find that both social learning and social utility channels have statistically and economically significant effects on investment decisions. Evidence from a follow-up survey reveals that social learning effects are greatest when the first (second) investor is financially sophisticated (financially unsophisticated); investors report updating their beliefs about asset quality after learning about their peer's revealed preference; and, they report motivations consistent with "keeping up with the Joneses" when learning about their peer's possession of the asset. These results can help shed light on the mechanisms underlying herding behavior in financial markets and peer effects in consumption and investment decisions.