Computing Bayes Factors by Combining Simulation and Asymptotic Approximations

DiCiccio, Thomas J.; Kass, Robert E.; Raftery, Adrian E.; Wasserman, Larry

doi:10.1080/01621459.1997.10474045

Cited by 520 publications

(112 citation statements)

References 22 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…Path sampling is hence an extension of bridge sampling, which generalizes importance sampling through the use of a single “bridge” density. In a comparative study on a variety of methods for computing Bayes factors, from Laplace approximation to bridge sampling, DiCiccio et al [30] show that bridge sampling typically provides an order of magnitude of improvement. Path sampling, which was not part of their study, has been demonstrated to yield even more dramatic improvement [10].…”

Section: Methodsmentioning

confidence: 99%

“…Up until the introduction of bridge sampling and path sampling, estimation methods in statistics often relied on the scheme of importance sampling, either using draws from an approximate density or from one of p i ( θ ). Theoretical [11] and empirical evidence [30,31] provided in the context of bridge sampling, show that substantial reductions of Monte Carlo errors can be achieved with little or minor increase in computational effort, by using draws from more than one p i ( θ ). The key idea is to use “bridge” densities to effectively shorten the distances among target densities, distances that are responsible for large Monte Carlo errors with the standard importance sampling methods.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Make the most of your samples: Bayes factor estimators for high-dimensional models of sequence evolution

2013

View full text Add to dashboard Cite

BackgroundAccurate model comparison requires extensive computation times, especially for parameter-rich models of sequence evolution. In the Bayesian framework, model selection is typically performed through the evaluation of a Bayes factor, the ratio of two marginal likelihoods (one for each model). Recently introduced techniques to estimate (log) marginal likelihoods, such as path sampling and stepping-stone sampling, offer increased accuracy over the traditional harmonic mean estimator at an increased computational cost. Most often, each model’s marginal likelihood will be estimated individually, which leads the resulting Bayes factor to suffer from errors associated with each of these independent estimation processes.ResultsWe here assess the original ‘model-switch’ path sampling approach for direct Bayes factor estimation in phylogenetics, as well as an extension that uses more samples, to construct a direct path between two competing models, thereby eliminating the need to calculate each model’s marginal likelihood independently. Further, we provide a competing Bayes factor estimator using an adaptation of the recently introduced stepping-stone sampling algorithm and set out to determine appropriate settings for accurately calculating such Bayes factors, with context-dependent evolutionary models as an example. While we show that modest efforts are required to roughly identify the increase in model fit, only drastically increased computation times ensure the accuracy needed to detect more subtle details of the evolutionary process.ConclusionsWe show that our adaptation of stepping-stone sampling for direct Bayes factor calculation outperforms the original path sampling approach as well as an extension that exploits more samples. Our proposed approach for Bayes factor estimation also has preferable statistical properties over the use of individual marginal likelihood estimates for both models under comparison. Assuming a sigmoid function to determine the path between two competing models, we provide evidence that a single well-chosen sigmoid shape value requires less computational efforts in order to approximate the true value of the (log) Bayes factor compared to the original approach. We show that the (log) Bayes factors calculated using path sampling and stepping-stone sampling differ drastically from those estimated using either of the harmonic mean estimators, supporting earlier claims that the latter systematically overestimate the performance of high-dimensional models, which we show can lead to erroneous conclusions. Based on our results, we argue that highly accurate estimation of differences in model fit for high-dimensional models requires much more computational effort than suggested in recent studies on marginal likelihood estimation.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Make the most of your samples: Bayes factor estimators for high-dimensional models of sequence evolution

2013

View full text Add to dashboard Cite

show abstract

“…The integral is then approximated by . For additional information on the Laplace method and other methods for Bayes factor approximation, see DiCiccio et al (1997).…”

Section: Methodsmentioning

confidence: 99%

Bayesian Adaptive Regression Splines for Hierarchical Data

Bigelow

Dunson

2007

Biometrics

View full text Add to dashboard Cite

This article considers methodology for hierarchical functional data analysis, motivated by studies of reproductive hormone profiles in the menstrual cycle. Current methods standardize the cycle lengths and ignore the timing of ovulation within the cycle, both of which are biologically informative. Methods are needed that avoid standardization, while flexibly incorporating information on covariates and the timing of reference events, such as ovulation and onset of menses. In addition, it is necessary to account for within-woman dependency when data are collected for multiple cycles. We propose an approach based on a hierarchical generalization of Bayesian multivariate adaptive regression splines. Our formulation allows for an unknown set of basis functions characterizing the population-averaged and woman-specific trajectories in relation to covariates. A reversible jump Markov chain Monte Carlo algorithm is developed for posterior computation. Applying the methods to data from the North Carolina Early Pregnancy Study, we investigate differences in urinary progesterone profiles between conception and nonconception cycles.

show abstract

“…In practice, the first term of the generalized Savage–Dickey density ratio is computed exactly as for the naïve Savage–Dickey ratio by drawing posterior samples from the full model

H_{1}

, which are then used to approximate the marginal posterior density at θ 0 . In a second step, the approximation of the correction term requires drawing posterior samples ψ ( t ) ( t = 1, … , T ) from the nested model with the equality constraint θ = θ 0 (Diciccio, Kass, Raftery, & Wasserman, ). These samples are then used to obtain a Monte Carlo estimate of the expectation,

\overset{true^}{normalE} false[\cdot false] = \frac{1}{T} {false\sum}_{t = 1}^{T} \frac{{normalπ}_{0} false(ψ^{false(t false)} false)}{{normalπ}_{1} false(ψ^{false(t false)} false| boldθ = θ_{0} false)} .

…”

Section: The Generalized Savage–dickey Density Ratiomentioning

confidence: 99%

A caveat on the Savage–Dickey density ratio: The case of computing Bayes factors for regression parameters

Heck

2018

Brit J Math & Statis

View full text Add to dashboard Cite

The Savage-Dickey density ratio is a simple method for computing the Bayes factor for an equality constraint on one or more parameters of a statistical model. In regression analysis, this includes the important scenario of testing whether one or more of the covariates have an effect on the dependent variable. However, the Savage-Dickey ratio only provides the correct Bayes factor if the prior distribution of the nuisance parameters under the nested model is identical to the conditional prior under the full model given the equality constraint. This condition is violated for multiple regression models with a Jeffreys-Zellner-Siow prior, which is often used as a default prior in psychology. Besides linear regression models, the limitation of the Savage-Dickey ratio is especially relevant when analytical solutions for the Bayes factor are not available. This is the case for generalized linear models, non-linear models, or cognitive process models with regression extensions. As a remedy, the correct Bayes factor can be computed using a generalized version of the Savage-Dickey density ratio.

show abstract

Computing Bayes Factors by Combining Simulation and Asymptotic Approximations

Cited by 520 publications

References 22 publications

Make the most of your samples: Bayes factor estimators for high-dimensional models of sequence evolution

Make the most of your samples: Bayes factor estimators for high-dimensional models of sequence evolution

Bayesian Adaptive Regression Splines for Hierarchical Data

A caveat on the Savage–Dickey density ratio: The case of computing Bayes factors for regression parameters

Contact Info

Product

Resources

About