Low power, type II errors, and other statistical problems in recent cardiovascular research

Abstract

Frequently in biomedical literature, measurements are considered "not statistically different" if a statistical test fails to achieve a P value that is < or = 0.05. This conclusion may be misleading because the size of each group is too small or the variability is large, and a type II error (false negative) is committed. In this study, we examined the probabilities of detecting a real difference (power) and type II errors in unpaired t-tests in Volumes 246 and 266 of the American Journal of Physiology: Heart and Circulatory Physiology. In addition, we examined all articles for other statistical errors. The median power of the t-tests was similar in these volumes (approximately 0.55 and approximately 0.92 to detect a 20% and a 50% change, respectively). In both volumes, approximately 80% of the studies with nonsignificant unpaired t-tests contained at least one t-test with a type II error probability > 0.30. Our findings suggest that low power and a high incidence of type II errors are common problems in this journal. In addition, the presentation of statistics was often vague, t-tests were misused frequently, and assumptions for inferential statistics usually were not mentioned or examined.

Low-intensity interval exercise training attenuates coronary vascular dysfunction and preserves Ca2+-sensitive K+ current in miniature swine with LV hypertrophy

“…Within-group comparisons were made using paired samples t-test. All data are means Ϯ SE, and significance is reported at P Ͻ 0.10 and P Ͻ 0.05 levels (12,52).…”

Low-intensity exercise training delays onset of decompensated heart failure in spontaneously hypertensive heart failure rats

“…Urine proteinuria and ANP levels were analyzed by using a repeated measures ANOVA. All data are means Ϯ SE, and significance is reported at P Ͻ 0.10, P Ͻ 0.05, and P Ͻ 0.01 levels (10,70).…”

Cardiac troponin T mutations result in allele-specific phenotypes in a mouse model for hypertrophic cardiomyopathy

“…In both types of analyses, statistical differences were identified at the P < 0.05 and P < 0.10 levels. The latter significance criterion was used following the principles described by Williams et al (27). Briefly, examination of data at the P < 0.10 level reduces the probability of committing a type II error (accepting the null hypothesis when it is false), and may be more appropriate for exploratory (rather than confirmatory) investigation.…”