Reply to Christensen and Christensen and to Malter: Pitfalls of erroneous analyses of hurricanes names

Jung, Kiju; Shavitt, Sharon; Viswanathan, Madhu; Hilbe, Joseph M.

doi:10.1073/pnas.1411652111

Cited by 10 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One reports that hurricanes with more feminine names have caused more deaths (Jung et al 2014a). We selected this paper because it led to an intense debate about the proper way to analyze the underlying data (Jung et al 2014a, Malter 2014, Maley 2014, Bakkensen and Larson 2014, Christensen and Christensen 2014, Jung et al 2014b, providing an opportunity to assess the extent to which specification-curve analysis could aid such debates. The second article reports a field experiment examining racial discrimination in the job market .…”

Section: Specification Curvementioning

confidence: 99%

Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications

2015

View full text Add to dashboard Cite

Empirical results often hinge on data analytic decisions that are simultaneously defensible, arbitrary, and motivated. To mitigate this problem we introduce Specification-Curve Analysis, which consists of three steps: (i) identifying the set of theoretically justified, statistically valid, and non-redundant analytic specifications, (ii) displaying alternative results graphically, allowing the identification of decisions producing different results, and (iii) conducting statistical tests to determine whether as a whole results are inconsistent with the null hypothesis. We illustrate its use by applying it to three published findings. One proves robust, one weak, one not robust at all. Abstract: Empirical results often hinge on data analytic decisions that are simultaneously defensible, arbitrary, and motivated. To mitigate this problem we introduce Specification-Curve Analysis, which consists of three steps: (i) identifying the set of theoretically justified, statistically valid, and non-redundant analytic specifications, (ii) displaying alternative results graphically, allowing the identification of decisions producing different results, and (iii) conducting statistical tests to determine whether as a whole results are inconsistent with the null hypothesis. We illustrate its use by applying it to three published findings. One proves robust, one weak, one not robust at all.

show abstract

Section: Specification Curvementioning

confidence: 99%

Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications

2015

View full text Add to dashboard Cite

show abstract

“…Yet, by conducting analyses across pre-processing pathways and comparing test statistics of the observed data with test statistics of datasets where the null hypothesis is true, we found that the number of significant pre-processing pathways is not different from what would be expected if the null hypothesis were true. That means, specification curve analysis has the potential to reduce the proportion of false positive findings in the scientific literature and shorten discussions about optimal analyses strategies (e.g., Christensen & Christensen, 2014;Jung et al, 2014aJung et al, , 2014bMalter, 2014;Munoz & Young, 2018;Simonsohn et al, 2020). Notably, most effects in the present study were robust to differences in data pre-processing: Either all 126 pathways indicated a significant result, or no pathway did, increasing confidence in the veracity of our findings.…”

Section: Discussionmentioning

confidence: 63%

The Effect of Face Masks on the Stereotype Effect in Emotion Perception

Primbs¹,

Rinck²,

Holland³

et al. 2022

Preprint

View full text Add to dashboard Cite

The accurate and swift decoding of emotional expressions from faces is fundamental for social communication. Yet, emotion perception is prone to error. For example, the ease with which emotions are perceived is affected by stereotypes (Bijlstra et al., 2010). Moreover, the introduction of face masks mandates in response to the Covid-19 pandemic additionally impedes accurate emotion perception by introducing ambiguity to the emotion perception process. Predictive coding frameworks of visual perception predict that in such situations of increased ambiguity of the sensory input (i.e., faces with masks), people increasingly rely on their prior beliefs (i.e., their stereotypes). Using specification curve analysis, we tested this prediction across two experiments, featuring different social categories (Study 1: Gender; Study 2: Ethnicity) and corresponding emotion stereotypes. We found no evidence that face masks increase reliance on prior stereotypes. In contrast, in Study 1 (but not in Study 2), we found preliminary evidence that face masks decrease reliance on prior stereotypes. We discuss these findings in relation to predictive coding frameworks and dual process models and emphasize the need for up-to-date analytic methods in social cognition research.

show abstract

“…At the same time, the Jung et al (2014a) paper triggered a daisy chain of critical, published letters to the editor (Bakkensen & Larson, 2014; Christensen & Christensen, 2014; Maley, 2014; Malter, 2014), along with published author replies and rebuttals (Jung et al, 2014b, 2014c, 2014d), all offering discrepant, and seemingly irreconcilable, views on which hurricane data to include and how to analyze them. Simonsohn et al (2015) assembled all these views (or, alternative specifications for data analysis) combinatorially, showed that this yielded 1,728 different ways to analyze more or less the same underlying data set, and further showed that the specific finding of female-named hurricanes being deadlier, as reported in Jung et al (2014a), belonged to a small subset of analyses (37 out of a total of 1,728 specifications, or 2.1%) which yielded a nominally significant result.…”

Section: Specification-curve and Multiverse-analysis Approaches To Th...mentioning

confidence: 99%

Which Data to Meta-Analyze, and How?

Voracek

Kossmeier

Tran

2019

Zeitschrift für Psychologie

View full text Add to dashboard Cite

Abstract. Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.

show abstract

Reply to Christensen and Christensen and to Malter: Pitfalls of erroneous analyses of hurricanes names

Cited by 10 publications

References 10 publications

Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications

Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications

The Effect of Face Masks on the Stereotype Effect in Emotion Perception

Which Data to Meta-Analyze, and How?

Contact Info

Product

Resources

About