Frequently in biomedical literature, measurements are considered "not statistically different" if a statistical test fails to achieve a P value that is < or = 0.05. This conclusion may be misleading because the size of each group is too small or the variability is large, and a type II error (false negative) is committed. In this study, we examined the probabilities of detecting a real difference (power) and type II errors in unpaired t-tests in Volumes 246 and 266 of the American Journal of Physiology: Heart and Circulatory Physiology. In addition, we examined all articles for other statistical errors. The median power of the t-tests was similar in these volumes (approximately 0.55 and approximately 0.92 to detect a 20% and a 50% change, respectively). In both volumes, approximately 80% of the studies with nonsignificant unpaired t-tests contained at least one t-test with a type II error probability > 0.30. Our findings suggest that low power and a high incidence of type II errors are common problems in this journal. In addition, the presentation of statistics was often vague, t-tests were misused frequently, and assumptions for inferential statistics usually were not mentioned or examined.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.