Using Epistemic Ratios to Evaluate Hypotheses: An Imprecision Penalty for Imprecise Hypotheses

Abstract

According to Bayesians, the null hypothesis significance-testing procedure is not deductively valid because it involves the retention or rejection of the null hypothesis under conditions where the posterior probability of that hypothesis is not known. Other criticisms are that this procedure is pointless and encourages imprecise hypotheses. However, according to non-Bayesians, there is no way of assigning a prior probability to the null hypothesis, and so Bayesian statistics do not work either. Consequently, no procedure has been accepted by both groups as providing a compelling reason to accept or reject hypotheses. The author aims to provide such a method. In the process, the author distinguishes between probability and epistemic estimation and argues that, although both are important in a science that is not completely deterministic, epistemic estimation is most relevant for hypothesis testing. Based on this analysis, the author proposes that hypotheses be evaluated via epistemic ratios and explores the implications of this proposal. One implication is that it is possible to encourage precise theorizing by imposing a penalty for imprecise hypotheses.

The Benefits of Applying Bayes’ Theorem in Medicine

“…Equation 14 makes clear that what we really need to know to reject the null hypothesis, which is , is a function of three variables of which (or p) is only one. The other two variables are the base rate probability of the null hypothesis (often termed the "prior" probability of the null hypothesis), and the probability of the finding given that the null hypothesis is not true.…”

“…Based on Equation 14, Figure 1 illustrates the negative mathematical consequences of using the NHSTP. There are much more extensive discussions that bring out additional problems [9,11,12,13,14].…”

“…The prior probability of the null hypothesis ranges from 0 to 1 Although many statisticians have recognized the basic logical invalidity of the NHSTP [9,11,12,13,14] it continues to be used in most sciences, including medicine. Some have tried to excuse the invalidity by on the one hand admitting the logical problem while on the other hand claiming that it does not matter too much because pvalues are strongly correlated with probabilities of null hypotheses [15].…”

“…Readers who are not interested in the derivation of Equation 19 can skip directly to it. I will use Equation 19 to solve the problem of the two physicians [14,17]. …”

“…To provide a fair playing field for the two hypotheses, it is customary to give them equal prior status so that the whole ratio commences at 1 [14]. However, this starting point can be updated as more data are collected, as I will show later.…”