Moving college health research forward: Reconsidering our reliance on statistical significance testing

Abstract

Understanding the unique health needs of college students and establishing best practices to address them depend, heavily, on the inherent quality and contribution of the research identifying these needs. College health-focused publications currently exemplify less than ideal statistical reporting practices. Specifically, college health practitioners and researchers continue to rely heavily upon null hypothesis significance testing (NHST) as the sole standard for effectiveness, validity, and/or replicability of scientific studies, even though NHST itself was not designed for such purposes. Herein we address the following questions: (a) What is NHST? (b) What are the inherent limitations of NHST? (c) What are recommended alternatives to NHST? and (d) How can editorial policies promote adopting NHST alternatives? Using college health data from the CORE 2011 Alcohol and Drug survey, we provide a heuristic example demonstrating how effect sizes do not suffer from the same limitations as NHST.

Remarkably reproducible psychological (memory) phenomena in the classroom: some evidence for generality from small-N research

“…Perhaps partly due to the latter erroneous replicability posture on the meaning of the venerated p value in extant psychology, there have been aforementioned failures in replication practices and reproducibility of important psychological findings (e.g., [ 5 , 6 ]) resulting in new efforts at promoting replications (see [ 20 , 42 ]), on one hand. On the other hand, NHST alternatives such as the new statistics recommending the use and reporting of effect sizes, confidence intervals, and meta-analyses [ 19 , 22 , 43 – 47 ] and Bayesian statistics [ 23 , 24 , 48 ] have been proffered. As Smith and Little [ 18 ] aptly observed, there has been an inadvertent demand for larger and larger samples in various journals as a matter of policy because of these efforts, to the detriment of the science we seek to advance particularly given the exemplary beneficial scientific features [ 45 ] of the alterative.…”