Reply to Maley: Yes, appropriate modeling of fatality counts confirms female hurricanes are deadlier

Jung, Kiju; Shavitt, Sharon; Viswanathan, Madhu; Hilbe, Joseph M.

doi:10.1073/pnas.1414111111

Cited by 4 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This analysis does not in itself mean that the hurricane study reported false positive results: That remains debatable (see Jung et al 2014c). We do not recommend automatically rejecting results that do not meet the threshold for robust.…”

Section: Are Female Hurricanes More Deadly? Testing Robustness In An mentioning

confidence: 96%

“…For our robustness analysis, we draw on a theoretically informed model space of alternative model ingredients. We incorporate all the published comments and rejoinders that emerged from the scholarly debate that followed the hurricane study and published in later issues of PNAS (Bakkensen and Larson 2014;Christensen and Christensen 2014;Jung et al 2014bJung et al , 2014cMaley 2014;Malter 2014). This includes different functional forms, treatment of outliers, a range of possible controls, alternative standard error calculations, efforts to address endogeneity, and different estimation commands.…”

Section: Are Female Hurricanes More Deadly? Testing Robustness In An mentioning

confidence: 99%

See 1 more Smart Citation

We Ran 9 Billion Regressions: Eliminating False Positives through Computational Model Robustness

Muñoz

Young

2018

Sociological Methodology

View full text Add to dashboard Cite

False positive findings are a growing problem in many research literatures. We argue that excessive false positives often stem from model uncertainty. There are many plausible ways of specifying a regression model, but researchers typically report only a few preferred estimates. This raises the concern that such research reveals only a small fraction of the possible results and may easily lead to nonrobust, false positive conclusions. It is often unclear how much the results are driven by model specification and how much the results would change if a different plausible model were used. Computational model robustness analysis addresses this challenge by estimating all possible models from a theoretically informed model space. We use large-scale random noise simulations to show (1) the problem of excess false positive errors under model uncertainty and (2) that computational robustness analysis can identify and eliminate false positives caused by model uncertainty. We also draw on a series of empirical applications to further illustrate issues of model uncertainty and estimate instability. Computational robustness analysis offers a method for relaxing modeling assumptions and improving the transparency of applied research.

show abstract

Section: Are Female Hurricanes More Deadly? Testing Robustness In An mentioning

confidence: 96%

Section: Are Female Hurricanes More Deadly? Testing Robustness In An mentioning

confidence: 99%

We Ran 9 Billion Regressions: Eliminating False Positives through Computational Model Robustness

Muñoz

Young

2018

Sociological Methodology

View full text Add to dashboard Cite

show abstract

“…At the same time, the Jung et al (2014a) paper triggered a daisy chain of critical, published letters to the editor (Bakkensen & Larson, 2014; Christensen & Christensen, 2014; Maley, 2014; Malter, 2014), along with published author replies and rebuttals (Jung et al, 2014b, 2014c, 2014d), all offering discrepant, and seemingly irreconcilable, views on which hurricane data to include and how to analyze them. Simonsohn et al (2015) assembled all these views (or, alternative specifications for data analysis) combinatorially, showed that this yielded 1,728 different ways to analyze more or less the same underlying data set, and further showed that the specific finding of female-named hurricanes being deadlier, as reported in Jung et al (2014a), belonged to a small subset of analyses (37 out of a total of 1,728 specifications, or 2.1%) which yielded a nominally significant result.…”

Section: Specification-curve and Multiverse-analysis Approaches To Th...mentioning

confidence: 99%

Which Data to Meta-Analyze, and How?

Voracek

Kossmeier

Tran

2019

Zeitschrift für Psychologie

View full text Add to dashboard Cite

Abstract. Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.

show abstract

Big data uncertainties

Maugis

2018

Journal of Forensic and Legal Medicine

View full text Add to dashboard Cite

Reply to Maley: Yes, appropriate modeling of fatality counts confirms female hurricanes are deadlier

Cited by 4 publications

References 6 publications

We Ran 9 Billion Regressions: Eliminating False Positives through Computational Model Robustness

We Ran 9 Billion Regressions: Eliminating False Positives through Computational Model Robustness

Which Data to Meta-Analyze, and How?

Big data uncertainties

Contact Info

Product

Resources

About