2016 Preprint# When null hypothesis significance testing is unsuitable for research: a reassessment

**Abstract:** Null hypothesis significance testing (NHST) has several shortcomings that are likely contributing factors behind the widely debated replication crisis of (cognitive) neuroscience, psychology, and biomedical science in general. We review these shortcomings and suggest that, after sustained negative experience, NHST should no longer be the default, dominant statistical practice of all biomedical and psychological research. If theoretical predictions are weak we should not rely on all or nothing hypothesis tests.…

Help me understand this report

View published versions

Search citation statements

Paper Sections

Select...

2

2

1

Citation Types

1

107

0

Year Published

2018

2022

Publication Types

Select...

7

2

Relationship

0

9

Authors

Journals

(108 citation statements)

(255 reference statements)

1

107

0

“…The ASA statement recommended: 'No single index should substitute for scientific reasoning' (10) -a viewpoint shared by other prominent investigators (45,46). In particular, Ioannidis and colleagues recently stressed monocultural training of biomedical scientists in statistical null-hypothesis testing as one reason behind frequent misuses of statistical methods (47).…”

confidence: 99%

“…The ASA statement recommended: 'No single index should substitute for scientific reasoning' (10) -a viewpoint shared by other prominent investigators (45,46). In particular, Ioannidis and colleagues recently stressed monocultural training of biomedical scientists in statistical null-hypothesis testing as one reason behind frequent misuses of statistical methods (47).…”

confidence: 99%

“…Historically, this type of deductive reasoning has often drawn on null hypothesis significance testing (NHST). The framework however is sometimes ill-suited and frequently misunderstood [17][18][19]. As an alternative to NHST, one may draw formal inference by means of false discovery rate (FDR), Bayesian posterior inference, or other tools [1, ch.…”

confidence: 99%

“…In contrast to conventional model fitting, Bayesian analysis allows for simple calculations of credible intervals (referred to as confidence intervals in the results) on derived parameters (e.g., parameters that are mathematical functions of fitted coefficients), offers simple construction of realistic hierarchical models (Gelman et al, 2013) and avoids a number of problems with conventional p-values derived from null hypothesis significance testing (Szucs and Ioannidis, 2017). Another advantage of Bayesian analysis is that it determines the probability that the model coefficients take on a particular value or range of values (McMillan and Cannon, 2019).…”

confidence: 99%