Marginal likelihood computation for model selection and hypothesis testing: an extensive review

Llorente, Fernando; Martino, Luca; Delgado, David; López‐Santiago, J.

doi:10.48550/arxiv.2005.08334

Cited by 23 publications

(53 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where (y|x) is the likelihood function, g(x) is the prior pdf, and Z(y) is the model evidence (a.k.a. marginal likelihood) which is a useful quantity in model selection problems [24]. For simplicity, in the following, we skip the dependence on y in p(x) = p(x|y) and Z = Z(y).…”

Section: Bayesian Inferencementioning

confidence: 99%

Optimality in Noisy Importance Sampling

Llorente,

Martino,

Read

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Many applications in signal processing and machine learning require the study of probability density functions (pdfs) that can only be accessed through noisy evaluations. In this work, we analyze the noisy importance sampling (IS), i.e., IS working with noisy evaluations of the target density. We present the general framework and derive optimal proposal densities for noisy IS estimators. The optimal proposals incorporate the information of the variance of the noisy realizations, proposing points in regions where the noise power is higher. We also compare the use of the optimal proposals with previous optimality approaches considered in a noisy IS framework.

show abstract

Section: Bayesian Inferencementioning

confidence: 99%

Optimality in Noisy Importance Sampling

Llorente,

Martino,

Read

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Then, in the second part (Section 5), we also estimate the marginal posterior p(σ|y). Finally, using (12), we can obtain a final approximation of the complete posterior p(θ, σ|y). Estimations of Z(σ) and Z are also obtained.…”

Section: Problem Statementmentioning

confidence: 99%

“…However, the user should decide a temperature schedule, i.e., a decreasing rule for the scale parameter, which is usually chosen in an heuristic way. In the literature, the tempering procedure has gained a particular attention for the estimation of the marginal likelihood (a.k.a., Bayesian model evidence) [9,11,12]. Furthermore, the joint inference of parameters (denoted as θ) of observation models, f(θ), and scale parameters of the likelihood function (that, in the scalar case, is usually denoted as σ) can be a hard task.…”

Section: Introductionmentioning

confidence: 99%

Automatic Tempered Posterior Distributions for Bayesian Inversion Problems

et al. 2021

Self Cite

View full text Add to dashboard Cite

We propose a novel adaptive importance sampling scheme for Bayesian inversion problems where the inference of the variables of interest and the power of the data noise are carried out using distinct (but interacting) methods. More specifically, we consider a Bayesian analysis for the variables of interest (i.e., the parameters of the model to invert), whereas we employ a maximum likelihood approach for the estimation of the noise power. The whole technique is implemented by means of an iterative procedure with alternating sampling and optimization steps. Moreover, the noise power is also used as a tempered parameter for the posterior distribution of the the variables of interest. Therefore, a sequence of tempered posterior densities is generated, where the tempered parameter is automatically selected according to the current estimate of the noise power. A complete Bayesian study over the model parameters and the scale parameter can also be performed. Numerical experiments show the benefits of the proposed approach.

show abstract

“…Finally, in order to aid truly Bayesian model selection of priors, it would be useful to be able to estimate marginal likelihoods from our Markov chains. This is generally a challenging problem, but there are a few promising solutions [20,21]. With these estimates, it could even be possible to learn useful BNN priors entirely from scratch [22].…”

Section: Limitationsmentioning

confidence: 99%

BNNpriors: A library for Bayesian neural network inference with different prior distributions

Fortuin,

Garriga-Alonso,

van der Wilk

et al. 2021

Preprint

View full text Add to dashboard Cite

Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance. However, it remains challenging to choose a good prior distribution over their weights. While isotropic Gaussian priors are often chosen in practice due to their simplicity, they do not reflect our true prior beliefs well and can lead to suboptimal performance. Our new library, BNNpriors, enables state-of-the-art Markov Chain Monte Carlo inference on Bayesian neural networks with a wide range of predefined priors, including heavy-tailed ones, hierarchical ones, and mixture priors. Moreover, it follows a modular approach that eases the design and implementation of new custom priors. It has facilitated foundational discoveries on the nature of the cold posterior effect in Bayesian neural networks and will hopefully catalyze future research as well as practical applications in this area.

show abstract

Marginal likelihood computation for model selection and hypothesis testing: an extensive review

Cited by 23 publications

References 60 publications

Optimality in Noisy Importance Sampling

Optimality in Noisy Importance Sampling

Automatic Tempered Posterior Distributions for Bayesian Inversion Problems

BNNpriors: A library for Bayesian neural network inference with different prior distributions

Contact Info

Product

Resources

About