Does Regression Discontinuity Design Work? Evidence from Random Election Outcomes

Hyytinen, Ari; Meriläinen, Jaakko; Saarimaa, Tuukka; Toivanen, Otto; Tukiainen, Janne

doi:10.2139/ssrn.2536286

Cited by 9 publications

(8 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Six studies provided us with standardized RD bias estimates directly or with the pooled standard deviation of the original study outcome necessary to calculate it directly (Aiken et al., ; Gleason, Resch, & Berk, ; Kisbu‐Sakarya, Cook, & Tang, in press; Nickerson, ; Shadish et al., ; Tang, Cook, & Kisbu‐Sakarya, forthcoming). Four studies used binary outcomes (Barrera‐Osorio, Filmer, & McIntyre, ; Berk et al., ; Green et al., ; Hyytinen et al., ). While it is common to use log odds ratios for binary outcomes in meta‐analyses, for ease of interpretation we stuck with mean differences of the binary outcomes and calculated treatment and control group standard deviations for them as the square roots of their variances.…”

Section: Methods For Meta‐analyzing Wsc Estimatesmentioning

confidence: 99%

The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons

Chaplin¹,

Cook²,

Zurovac³

et al. 2018

J Policy Anal Manage

View full text Add to dashboard Cite

Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD's internal validity across 15 studies. Each of them assesses the correspondence between causal estimates from an RD study and a randomized control trial (RCT) when the estimates are made at the same cutoff point where they should not differ asymptotically. However, statistical error, imperfect design implementation, and a plethora of different possible analysis options, mean that they might nonetheless differ. We test whether they do, assuming that the bias potential is greater with RDs than RCTs. A second purpose of this paper is to investigate the external validity of RD by exploring how the size of the bias estimates varies across the 15 studies, for they differ in their settings, interventions, analyses, and implementation details. Both Bayesian and frequentist meta‐analysis methods show that the RD bias is below 0.01 standard deviations on average, indicating RD's high internal validity. When the study‐specific estimates are shrunken to capitalize on the information the other studies provide, all the RD causal estimates fall within 0.07 standard deviations of their RCT counterparts, now indicating high external validity. With unshrunken estimates, the mean RD bias is still essentially zero, but the distribution of RD bias estimates is less tight, especially with smaller samples and when parametric RD analyses are used.

show abstract

Section: Methods For Meta‐analyzing Wsc Estimatesmentioning

confidence: 99%

The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons

Chaplin¹,

Cook²,

Zurovac³

et al. 2018

J Policy Anal Manage

View full text Add to dashboard Cite

show abstract

“…See Folke (2014); Freier and Odendahl (2012) ;Hyytinen, Meriläinen, Saarimaa, Toivanen, and Tukiainen (2014); close elections it is as good as random which political bloc gets a seat majority, and that this in turn strongly determines which party gets the governorship.…”

Section: Introductionmentioning

confidence: 99%

Local favoritism in at-large proportional representation systems

Fiva

Halse

2016

Journal of Public Economics

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Pork barrel spending is typically attributed to the strategic behavior of political elites hoping to be electorally rewarded by voters residing in their districts. Such behavior is expected to depend on the incentives imposed by the electoral system. We estimate the causal effect of local representation in a closed list proportional representation system where individual candidates have no clear electoral incentive to favor their hometown. Using data from Norwegian regional governments, we still find a hometown bias. We document that municipalities with a representative on the regional council from the same party as the governor tend to obtain more funding for local investments. In an extension we show that citizens also tend to vote more often for parties with a governor candidate from their own hometown, consistent with an expectation of particularistic benefits. Both effects are driven by municipalities with below median population size. Terms of use: Documents inJEL-Code: D720, H720, H770.

show abstract

“…Estimation is delicate because the functional forms are unknown, and one is estimating the trend at an endpoint (the cutoff) of measure zero. The example in Hyytinen et al (2015) illustrates how inferences in RD designs are sensitive to the method used to estimate the continuous functions on both sides of the cutoff even when sample sizes are not small, and the design is valid by construction. More generally, the properties of RD estimation and inference based on local polynomial methods depend crucially on the bandwidth choice; for example, the commonly used mean-squared-error optimal bandwidth, though valid for point estimation, is too large for inference and requires either undersmoothing or robust methods to yield valid confidence intervals (Calonico, Cattaneo, and Titiunik 2014).…”

Section: Discussionmentioning

confidence: 99%

“…Note that this setup also assumes that the score X i collected by the researcher is the second stage score for all students, so that the treatment rule 1(X i ≤ 50) correctly distinguishes treated and control students. Such a two-stage rule is of course artificial, although a second-stage randomization has been used in some actual RD settings (see, e.g., Hyytinen et al 2015). 4…”

Section: Local Randomization Of Score or Treatment?mentioning

confidence: 99%

“…The interpretation of RD designs as local experiments developed by Lee (2008) has been very influential, both conceptually and practically. Among other things, it established the need to provide falsification tests based on predetermined covariates just as one would do in the analysis of experiments, a practice that has now been widely adopted and has increased the credibility of countless RD applications (e.g., Caughey and Sekhon 2011;Eggers et al 2014;Hyytinen et al 2015). Moreover, it provided an intuitive interpretation of the RD parameter that allowed researchers to think about treatment and control groups instead of an effect at a single point-the cutoff-where there are effectively no observations.…”

mentioning

confidence: 99%

See 1 more Smart Citation

On Interpreting the Regression Discontinuity Design as a Local Experiment

Sekhon

Titiunik²

2017

Advances in Econometrics

View full text Add to dashboard Cite

We discuss the two most popular frameworks for identification, estimation and inference in regression discontinuity (RD) designs: the continuity-based framework, where the conditional expectations of the potential outcomes are assumed to be continuous functions of the score at the cutoff, and the local randomization framework, where the treatment assignment is assumed to be as good as randomized in a neighborhood around the cutoff. Using various examples, we show that (i) assuming random assignment of the RD running variable in a neighborhood of the cutoff neither implies that the potential outcomes and the treatment are statistically independent nor that the potential outcomes are unrelated to the running variable in this neighborhood, and (ii) assuming local independence between the potential outcomes and the treatment does not imply the exclusion restriction that the score affects the outcomes only through the treatment indicator. Our discussion highlights key distinctions between "locally randomized" RD designs and real experiments, including that statistical independence and random assignment are conceptually different in RD contexts, and that the RD treatment assignment rule places no restrictions on how the score and potential outcomes are related. Our findings imply that the methods for RD estimation, inference and falsification used in practice will necessarily be different (both in formal properties and in interpretation) according to which of the two frameworks is invoked.

show abstract

Does Regression Discontinuity Design Work? Evidence from Random Election Outcomes

Cited by 9 publications

References 34 publications

The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons

The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons

Local favoritism in at-large proportional representation systems

On Interpreting the Regression Discontinuity Design as a Local Experiment

Contact Info

Product

Resources

About