Comparative politics and the synthetic control method revisited: a note on Abadie et al. (2015)

Klößner, Stefan; Kaul, Ashok; Pfeifer, Gregor; Schieler, Manuel

doi:10.1186/s41937-017-0004-9

Cited by 42 publications

(31 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we analyze the possibilities for specification searching and provide our recommendations in a series of empirical applications. C 2020 by the Association for Public Policy Analysis and Management on behalf of the Association for Public Policy Analysis and Management variables, the choice of how to split the pre-treatment periods into training and validation periods, and even the choice of software and data-sorting criteria (see Klößner et al, 2017, for details on this last point). Therefore, our results should be seen as a lower bound on the possibilities for specification searching in SC applications.…”

mentioning

confidence: 99%

Cherry Picking with Synthetic Controls

Ferman¹,

Pinto²,

Possebom³

2020

J Policy Anal Manage

160

150

View full text Add to dashboard Cite

We evaluate whether a lack of guidance on how to choose the matching variables used in the Synthetic Control (SC) estimator creates specification-searching opportunities. We provide theoretical results showing that specification-searching opportunities are asymptotically irrelevant if we restrict to a subset of SC specifications. However, based on Monte Carlo simulations and simulations with real datasets, we show significant room for specification searching when the number of pre-treatment periods is in line with common SC applications, and when alternative specifications commonly used in SC applications are also considered. This suggests that such lack of guidance generates a substantial level of discretion in the choice of the comparison units in SC applications, undermining one of the advantages of the method. We provide recommendations to limit the possibilities for specification searching in the SC method. Finally, we analyze the possibilities for specification searching and provide our recommendations in a series of empirical applications. C 2020 by the Association for Public Policy Analysis and Management on behalf of the Association for Public Policy Analysis and Management variables, the choice of how to split the pre-treatment periods into training and validation periods, and even the choice of software and data-sorting criteria (see Klößner et al., 2017, for details on this last point). Therefore, our results should be seen as a lower bound on the possibilities for specification searching in SC applications. We focus on the choice of pre-treatment outcome lags, rather than on the inclusion of covariates, because it is possible to systematically analyze the inclusion of pre-intervention outcomes lags in a way that encompasses all applications, while covariates may differ in complex ways from one application to another. We consider the possibility of specification searching in the decision to include covariates in our empirical applications. 6 Appendices are available at the end of this article as it appears in JPAM online. Go to the publisher's website and use the search engine to locate the article at http://onlinelibrary.wiley.com. 7 In Appendix B, we also consider specifications that use time-invariant covariates as predictors, in addition to functions of the pre-treatment outcomes. All results remain similar. This package solves the nested minimization problem described by equations (1) and (2). We specify the optimization method to be BFGS only and use optimization routine Low Rank Quadratic Programming when Interior Point optimization routine does not converge. 24 When we adopt this decision rule in our MC simulations, then the probability of rejecting the null at 5 percent for all specifications is lower than 1 percent in all scenarios. If we discard specifications 6 and 7, then this rejection rate ranges from 1 percent when T 0 = 12 to 2.8 percent when T 0 = 400. 25 Following the best practices in terms of transparency and replicability, Hainmueller (2014) made their dataset and replication ...

show abstract

mentioning

confidence: 99%

Cherry Picking with Synthetic Controls

Ferman¹,

Pinto²,

Possebom³

2020

J Policy Anal Manage

160

150

View full text Add to dashboard Cite

show abstract

“…In a second step, the just-selected V is used as an input to find a new set of optimal country weights W that minimises the differences in predictor variables between the treated unit and its synthetic control over the validation period. In spite of some concerns with this approach (seeKlößner et al, 2017), we have also run, as a robustness check, all our synthetic control experiments using cross-validation. The estimated FCL effects are broadly similar to those presented in Sections 4.3.1 and 4.3.2.…”

mentioning

confidence: 99%

The IMF and precautionary lending: An empirical evaluation of the selectivity and effectiveness of the Flexible Credit Line

Essers

Ide

2019

Journal of International Money and Finance

View full text Add to dashboard Cite

The purpose of these working papers is to promote the circulation of research results (Research Series) and analytical studies (Documents Series) made within the National Bank of Belgium or presented by external economists in seminars, conferences and conventions organised by the Bank. The aim is therefore to provide a platform for discussion. The opinions expressed are strictly those of the authors and do not necessarily reflect the views of the National Bank of Belgium.

show abstract

“…As Klößner, Kaul, Pfeifer, and Schieler () show, one should be careful with interpreting such

v

weights with respect to “importance” since they are not uniquely defined. If we follow M. Becker, Klößner, and Pfeifer () and apply a cross‐validation to determine predictor weights

v

, we arrive at results fully in line with our approach here.…”

mentioning

confidence: 99%

Illuminating the World Cup effect: Night lights evidence from South Africa

Pfeifer

Wahl

Marczak

2018

Journal of Regional Science

Self Cite

View full text Add to dashboard Cite

This paper evaluates the economic impact of the $14 billion preparatory infrastructure investments for the 2010 FIFA World Cup in South Africa. We use satellite data on night light luminosity at municipality and electoral district level as a proxy for economic activity, applying synthetic control methods for estimation. For the average World Cup municipality, we find significantly positive, short-run effects before the tournament, corresponding to a reduction of unemployment by 1.3 percentage points. At the electoral district level, we reveal distinct effect heterogeneity, where especially investments in transport infrastructure are shown to have long-lasting, positive effects, particularly in rural areas.

show abstract

Comparative politics and the synthetic control method revisited: a note on Abadie et al. (2015)

Cited by 42 publications

References 13 publications

Cherry Picking with Synthetic Controls

Cherry Picking with Synthetic Controls

The IMF and precautionary lending: An empirical evaluation of the selectivity and effectiveness of the Flexible Credit Line

Illuminating the World Cup effect: Night lights evidence from South Africa

Contact Info

Product

Resources

About