Star Wars: The Empirics Strike Back

Brodeur, Abel; Lé, Mathias; Sangnier, Marc; Zylberberg, Yanos

doi:10.2139/ssrn.2089580

Cited by 48 publications

(58 citation statements)

References 31 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our first yardstick will be the number of variables which are 'robust', those where the posterior inclusion probability exceeds the prior inclusion probability; These numbers are reported in Table 4. 12 We report two sets of results, one based on the 10 This perspective acknowledges that, in the growth setting, the true data generating process is unlikely to be among the models considered, even when the number of candidate models is very large. This raises the question of whether AIC-based methods for model averaging are worth exploring; see Anderson (2002, 2004) and Claeskens and Hjort (2008).…”

Section: Resultsmentioning

confidence: 99%

“…11 In much the same way, panel data estimation of the Solow model with country fixed effects (for example, Islam 1995) uses within-country variation to identify the same structural parameters as the cross-section study of Mankiw et al (1992). 12 These and the other BMA results in the paper were obtained using the BMS software due to Feldkircher and Zeugner. Taken together, these results point clearly in one direction.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Growth econometrics for agnostics and true believers

Rockey

Temple

2016

European Economic Review

View full text Add to dashboard Cite

The issue of model uncertainty is central to the empirical study of economic growth. Many recent papers use Bayesian Model Averaging to address model uncertainty, but Ciccone and Jarociński (2010) have questioned the approach on theoretical and empirical grounds. They argue that a standard 'agnostic' approach is too sensitive to small changes in the dependent variable, such as those associated with different vintages of the Penn World Table (PWT). This paper revisits their theoretical arguments and empirical illustration, drawing on more recent vintages of the PWT, and introducing an approach that limits the degree of agnosticism.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Growth econometrics for agnostics and true believers

Rockey

Temple

2016

European Economic Review

View full text Add to dashboard Cite

show abstract

“…These include selectively removing outliers and trying different testing procedures until results are significant (Abelson, 1995, p.55). Such practices go by various names such as p-hacking, torturing data, data dredging, or researcher degrees of freedom (Nuzzo, 2014;Lakens et al, 2014;Simmons et al, 2011;Brodeur et al, 2012;Gelman and Loken, 2013). They differ from the legitimate practice of exploratory data analysis (Tukey, 1980) because their goal is to obtain the results one wishes for, not to learn or to inform.…”

Section: Issues Regarding P-hackingmentioning

confidence: 99%

Fair Statistical Communication in HCI

Dragicevic

2016

Human–Computer Interaction Series

258

247

View full text Add to dashboard Cite

Statistics are tools to help end users accomplish their task. In research, to be qualified as usable, statistical tools should help researchers advance scientific knowledge by supporting and promoting the effective communication of research findings. Yet areas such as human-computer interaction (HCI) have adopted toolsi.e., p-values and dichotomous testing procedures -that have proven to be poor at supporting these tasks. The abusive use of these procedures has been severely criticized in a range of disciplines for several decades, suggesting that tools should be blamed, not end users. This chapter explains in a non-technical manner why it would be beneficial for HCI to switch to an estimation approach, i.e., reporting informative charts with effect sizes and interval estimates, and offering nuanced interpretations of our results. Advice is offered on how to communicate our empirical results in a clear, accurate, and transparent way without using any tests or p-values.

show abstract

“…Second, biases can enter into the research process. For example, specification searching or publication bias can lead to an over-representation of positive effects (Brodeur et al, 2016;Vivalt, 2017a), while the "file-drawer problem" means that non-significant results often fail to see the light of day (Rosenthal, 1979). The third reason for low reproducibility -and the focus of this paperis statistical in nature.…”

Section: Introductionmentioning

confidence: 97%

How Often Should We Believe Positive Results? Assessing the Credibility of Research Findings in Development Economics

Coville¹,

Vivalt²

2017

Preprint

View full text Add to dashboard Cite

Under-powered studies combined with low prior beliefs about intervention effects increase the chances that a positive result is overstated. We collect prior beliefs about intervention impacts from 125 experts to estimate the false positive and false negative report probabilities (FPRP and FNRP) as well as Type S (sign) and Type M (magnitude) errors for studies in development economics.We find that the large majority of studies in our sample are generally credible.We discuss how more systematic collection and use of prior expectations could help strengthen the literature. *

show abstract

Star Wars: The Empirics Strike Back

Abstract: jected tests. Our interpretation is that researchers might be tempted to inflate the value of those just-rejected tests by choosing a "significant" specification. We propose a method to measure this residual and describe how it varies by article and author characteristics.

Cited by 48 publications

References 31 publications

Growth econometrics for agnostics and true believers

Growth econometrics for agnostics and true believers

Fair Statistical Communication in HCI

How Often Should We Believe Positive Results? Assessing the Credibility of Research Findings in Development Economics

Contact Info

Product

Resources

About