The Case Against Statistical Significance Testing, Revisited

Carver, Ronald P.

doi:10.1080/00220973.1993.10806591

Cited by 249 publications

(211 citation statements)

References 13 publications

Supporting

Mentioning

202

Contrasting

Unclassified

Order By: Relevance

“…Hence, ultimately any H 1 can be accepted, claiming support for any kind of theory. For example, in an amusing demonstration Carver (1993) …”

Section: Nhst In Sciences With and Without Exact Quantitative Predictmentioning

confidence: 99%

When null hypothesis significance testing is unsuitable for research: a reassessment

Szűcs

Ioannidis

2016

Preprint

107

View full text Add to dashboard Cite

Null hypothesis significance testing (NHST) has several shortcomings that are likely contributing factors behind the widely debated replication crisis of (cognitive) neuroscience, psychology, and biomedical science in general. We review these shortcomings and suggest that, after sustained negative experience, NHST should no longer be the default, dominant statistical practice of all biomedical and psychological research. If theoretical predictions are weak we should not rely on all or nothing hypothesis tests. Different inferential methods may be most suitable for different types of research questions. Whenever researchers use NHST they should justify its use, and publish pre-study power calculations and effect sizes, including negative findings. Hypothesis-testing studies should be pre-registered and optimally raw data published. The current statistics lite educational approach for students that has sustained the widespread, spurious use of NHST should be phased out.

show abstract

“…Hence, ultimately any H 1 can be accepted, claiming support for any kind of theory. For example, in an amusing demonstration Carver (1993) …”

Section: Nhst In Sciences With and Without Exact Quantitative Predictmentioning

confidence: 99%

When null hypothesis significance testing is unsuitable for research: a reassessment

Szűcs

Ioannidis

2016

Preprint

107

View full text Add to dashboard Cite

show abstract

“…Several authors over the past several years (e.g., Carver, 1993;Cohen, 1990;Schmidt, 1992) , 1997). In the present sample of studies the behavioral science researchers were either unaware of these recommendations or chose to ignore them.…”

Section: Assessment Of Validity Assumptionsmentioning

confidence: 99%

Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses

Keselman

Huberty²,

Lix³

et al. 1998

Review of Educational Research

480

279

View full text Add to dashboard Cite

Articles published in several prominent educational journals were examined to investigate the use of data-analysis tools by researchers in four research paradigms: between-subjects univariate designs, between-subjects multivariate designs, repeated measures designs, and covariance designs. In addition to examining specific details pertaining to the research design (e.g., sample size, group size equality/inequality) and methods employed for data analysis, we also catalogued whether: (a) validity assumptions were examined, (b) effect size indices were reported, (c) sample sizes were selected based on power considerations, and (d) appropriate textbooks and/or articles were cited to communicate the nature of the analyses that were performed. Our analyses imply that researchers rarely verify that validity assumptions are satisfied and accordingly typically use analyses that are nonrobust to assumption violations. In addition, researchers rarely report effect size statistics, nor do they routinely perform power analyses to determine sample size requirements. We offer many recommendations to rectify these shortcomings. Data Analytic Practices 3 Statistical Practises of Educational Researchers:An Analysis of Their ANOVA, MANOVA and ANCOVA Analyses It is well known that the volume of published educational research is increasing at a very rapid pace. As a consequence of the expansion of the field, qualitative and quantitative reviews of the literature are becoming more common. These reviews typically focus on summarizing the results of research in particular areas of scientific inquiry (e.g., academic achievement or English as a second language) as a means of highlighting important findings and identifying gaps in the literature. Less common, but equally important, are reviews that focus on the research process, that is, the methods by which a research topic is addressed, including research design and statistical analysis issues.Methodological research reviews have a long history (e.g., Edgington, 1964; Elmore & Woehlke, 1988 Goodwin & Goodwin, 1985a, 1985bWest, Carmody, & Stallings, 1983).One purpose of these reviews has been the identification of trends in data-analytic practice. The documentation of such trends has a two-fold purpose: (a) it can form the basis for recommending improvements in research practice, and (b) it can be used as a guide for the types of inferential procedures that should be taught in methodological courses, so that students have adequate skills to interpret the published literature of a discipline and to carry out their own projects.One consistent finding of methodological research reviews is that a substantial gap often exists between the inferential methods that are recommended in the statistical research literature and those techniques that are actually adopted by applied researchers (Goodwin & Goodwin, 1985b;Ridgeway, Dunston, & Qian, 1993). The practice of relying on traditional methods of analysis is, however, dangerous. The field of statistics is by no means static; improveme...

show abstract

“…Some classic references criticizing standard hypothesis testing include Boring (1919), Berkson (1938Berkson ( , 1942, Rozeboom (1960), Meehl (1967Meehl ( , 1978, and Carver (1978). More recently, some support the continued usage of significance testing (Abelson, 1997;Hagan, 1997Hagan, , 1998Harris, 1997;McLean & Ernest, 1998), while others desire a greater reliance on alternatives such as confidence intervals or effect sizes (Cohen, 1992(Cohen, , 1994Knapp, 1998Knapp, , 2002Meehl, 1997;Serlin, 2002;Thompson, 1998Thompson, , 2001Vacha-Haase, 2001), and still others advocate an outright ban on significance testing (Carver, 1993;Falk, 1998;Nix & Barnette, 1998;Schmidt & Hunter, 1997).…”

Section: Introductionmentioning

confidence: 99%

A Comparison Of Equivalence Testing In Combination With Hypothesis Testing And Effect Sizes

Mecklin

2003

J. Mod. App. Stat. Meth.

View full text Add to dashboard Cite

show abstract

The Case Against Statistical Significance Testing, Revisited

Cited by 249 publications

References 13 publications

When null hypothesis significance testing is unsuitable for research: a reassessment

When null hypothesis significance testing is unsuitable for research: a reassessment

Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses

A Comparison Of Equivalence Testing In Combination With Hypothesis Testing And Effect Sizes

Contact Info

Product

Resources

About