Sharpen statistical significance: Evidence thresholds and Bayes factors sharpened into Occam's razor

Bickel, David R.

doi:10.1002/sta4.215

Cited by 10 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Equation ( 24) suggests viewing σ κ B (x; σ) as the κ-sharpened Bayes factor, applicable regardless of the value of P (0). Under the ideal value of κ derived in Section 4, that simplicity adjustment, when coupled with an argument of Benjamin et al (2017), leads to 0.001 or 0.01 rather than 0.005 or 0.05 as the default p-value threshold of statistical significance (Bickel, 2019c).…”

Section: Adjusting Priors For the Simplicity Of Data Pdfsmentioning

confidence: 99%

See 1 more Smart Citation

An Explanatory Rationale for Priors Sharpened Into Occam’s Razors

Bickel¹

2020

Bayesian Anal.

Self Cite

View full text Add to dashboard Cite

In Bayesian statistics, if the distribution of the data is unknown, then each plausible distribution of the data is indexed by a parameter value, and the prior distribution of the parameter is specified. To the extent that more complicated data distributions tend to require more coincidences for their construction than simpler data distributions, default prior distributions should be transformed to assign additional prior probability or probability density to the parameter values that refer to simpler data distributions. The proposed transformation of the prior distribution relies on the entropy of each data distribution as the relevant measure of complexity. The transformation is derived from a few first principles and extended to stochastic processes.

show abstract

Section: Adjusting Priors For the Simplicity Of Data Pdfsmentioning

confidence: 99%

“…Another implication is that prior distributions that represent known physical variability do not require adjustments for simplicity (cf. Bickel, 2019c), for their probabilities are limiting relative frequencies that do not depend on the construction of systems.…”

Section: Infinite-θ Examplesmentioning

confidence: 99%

An Explanatory Rationale for Priors Sharpened Into Occam’s Razors

Bickel¹

2020

Bayesian Anal.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Different assumptions lead to different versions of B , the lower bound on the Bayes factor (Held and Ott, 2018). Instead of the version given by equation (3), this example uses B = e − z 2 , which Held and Ott (2016) call the universal lower bound, where z is the standard normal quantile of a one-sided p value testing ϑ = θ H 0 (Bickel, 2019g). Since that B may be too low to be a reasonable estimate of the Bayes factor B , it might require some kind of averaging with B , an upper bound on B .…”

Section: Estimating the Local False Discovery Ratementioning

confidence: 99%

Interval estimation, point estimation, and null hypothesis significance testing calibrated by an estimated posterior probability of the null hypothesis

Bickel

2021

Communications in Statistics - Theory and Methods

Self Cite

View full text Add to dashboard Cite

Much of the blame for failed attempts to replicate reports of scientific findings has been placed on ubiquitous and persistent misinterpretations of the p value. An increasingly popular solution is to transform a two-sided p value to a lower bound on a Bayes factor. Another solution is to interpret a one-sided p value as an approximate posterior probability.Combining the two solutions results in confidence intervals that are calibrated by an estimate of the posterior probability that the null hypothesis is true. The combination also provides a point estimate that is covered by the calibrated confidence interval at every level of confidence.Finally, the combination of solutions generates a two-sided p value that is calibrated by the estimate of the posterior probability of the null hypothesis. In the special case of a 50% prior probability of the null hypothesis and a simple lower bound on the Bayes factor, the calibrated two-sided p value is about (1 -abs(2.7 p ln p)) p + 2 abs(2.7 p ln p) for small p. The calibrations of confidence intervals, point estimates, and p values are proposed in an empirical Bayes framework without requiring multiple comparisons.

show abstract

“…To reduce failure to replicate, one solution suggested in the literature is the use of stricter evidential thresholds, possibly variable by discipline (Johnson, 2013; Goodman, 2016). Benjamin et al (2018) and Bickel (2019) advocate changing the standard threshold for significance from 0.05 to 0.005, or even 0.001, while Lakens et al (2018) recommend a case by case transparently justified choice, better if preregistered.…”

Section: Introductionmentioning

confidence: 99%

Likelihood, Replicability and Robbins' Confidence Sequences

Pace

Salvan

2019

Int Statistical Rev

View full text Add to dashboard Cite

The widely claimed replicability crisis in science may lead to revised standards of significance. The customary frequentist confidence intervals, calibrated through hypothetical repetitions of the experiment that is supposed to have produced the data at hand, rely on a feeble concept of replicability. In particular, contradictory conclusions may be reached when a substantial enlargement of the study is undertaken. To redefine statistical confidence in such a way that inferential conclusions are non-contradictory, with large enough probability, under enlargements of the sample, we give a new reading of a proposal dating back to the 60's, namely Robbins' confidence sequences. Directly bounding the probability of reaching, in the future, conclusions that contradict the current ones, Robbins' confidence sequences ensure a clear-cut form of replicability when inference is performed on accumulating data. Their main frequentist property is easy to understand and to prove. We show that Robbins' confidence sequences may be justified under various views of inference: they are likelihood-based, can incorporate prior information, and obey the strong likelihood principle. They are easy to compute, even when inference is on a parameter of interest, especially using a closed-form approximation from normal asymptotic theory.

show abstract

Sharpen statistical significance: Evidence thresholds and Bayes factors sharpened into Occam's razor

Cited by 10 publications

References 21 publications

An Explanatory Rationale for Priors Sharpened Into Occam’s Razors

An Explanatory Rationale for Priors Sharpened Into Occam’s Razors

Interval estimation, point estimation, and null hypothesis significance testing calibrated by an estimated posterior probability of the null hypothesis

Likelihood, Replicability and Robbins' Confidence Sequences

Contact Info

Product

Resources

About