A Unified Particle-Optimization Framework for Scalable Bayesian Sampling

Chen, Changyou; Zhang, Ruiyi; Wang, Wenlin; Li, Bai; Chen, Liqun

doi:10.48550/arxiv.1805.11659

Cited by 22 publications

(47 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the following discussion, we will call such problem (11) constrained Entropy Transport problem since we constrain the space for γ on P(R d × R d ).…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

“…Theorem 5 (Characterization of optimal distribution γ cET to problem (11) ). We assume supp(µ) = supp(ν) = R d .…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

“…We assume supp(µ) = supp(ν) = R d . Suppose γ cET solves the constrained Entropy Transport problem (11). Then there exist certain ϕ, ψ ∈ B(R d ; R) satisfying: ϕ(x) + ψ(y) ≤ c(x, y) for any x, y ∈ R d , such that:…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

“…We provide a direct proof of this theorem in Appendix A.2. We can compare the structure of solution γ cET to (11) with the solution γ OT to the Optimal Transport problem (6).…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

See 3 more Smart Citations

Approximating the Optimal Transport Plan via Particle-Evolving Method

Liu,

Sun,

Zha

2021

Preprint

View full text Add to dashboard Cite

Optimal transport (OT) provides powerful tools for comparing probability measures in various types. The Wasserstein distance which arises naturally from the idea of OT is widely used in many machine learning applications. Unfortunately, computing the Wasserstein distance between two continuous probability measures always suffers from heavy computational intractability. In this paper, we propose an innovative algorithm that iteratively evolves a particle system to match the optimal transport plan for two given continuous probability measures. The derivation of the algorithm is based on the construction of the gradient flow of an Entropy Transport Problem which could be naturally understood as a classical Wasserstein optimal transport problem with relaxed marginal constraints. The algorithm comes with theoretical analysis and empirical evidence.

show abstract

“…In the following discussion, we will call such problem (11) constrained Entropy Transport problem since we constrain the space for γ on P(R d × R d ).…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

“…Theorem 5 (Characterization of optimal distribution γ cET to problem (11) ). We assume supp(µ) = supp(ν) = R d .…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

“…We provide a direct proof of this theorem in Appendix A.2. We can compare the structure of solution γ cET to (11) with the solution γ OT to the Optimal Transport problem (6).…”

Section: Constrained Entropy Transport Problem and Some Of Its Proper...mentioning

confidence: 99%

See 2 more Smart Citations

Approximating the Optimal Transport Plan via Particle-Evolving Method

Liu,

Sun,

Zha

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…( 1) has no closed-form and gives rise to a combinatorial problem. Several approximation strategies have been proposed, including Maximum A Posteriori inference, Markov Chain Monte Carlo [3,4,7], Expectation Propagation [16,23] and Variational Inference [26]. A common approach is to draw samples θ k from the posterior p(θ|X, T ) or from an approximation q(θ), followed by Monte Carlo integration, yielding:…”

Section: Introductionmentioning

confidence: 99%

Is MC Dropout Bayesian?

Folgoc,

Baltatzis,

Desai

et al. 2021

Preprint

View full text Add to dashboard Cite

MC Dropout is a mainstream "free lunch" method in medical imaging for approximate Bayesian computations (ABC). Its appeal is to solve out-of-the-box the daunting task of ABC and uncertainty quantification in Neural Networks (NNs); to fall within the variational inference (VI) framework; and to propose a highly multimodal, faithful predictive posterior. We question the properties of MC Dropout for approximate inference, as in fact MC Dropout changes the Bayesian model; its predictive posterior assigns 0 probability to the true model on closed-form benchmarks; the multimodality of its predictive posterior is not a property of the true predictive posterior but a design artefact. To address the need for VI on arbitrary models, we share a generic VI engine within the pytorch framework. The code includes a carefully designed implementation of structured (diagonal plus low-rank) multivariate normal variational families, and mixtures thereof. It is intended as a go-to nofree-lunch approach, addressing shortcomings of mean-field VI with an adjustable trade-off between expressivity and computational complexity.

show abstract

Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior

Cho,

Bae,

Shin

et al. 2024

AAAI

View full text Add to dashboard Cite

Recent vision-language pre-trained (VLP) models have become the backbone for many downstream tasks, but they are utilized as frozen model without learning. Prompt learning is a method to improve the pre-trained VLP model by adding a learnable context vector to the inputs of the text encoder. In a few-shot learning scenario of the downstream task, MLE training can lead the context vector to over-fit dominant image features in the training data. This overfitting can potentially harm the generalization ability, especially in the presence of a distribution shift between the training and test dataset. This paper presents a Bayesian-based framework of prompt tuning, which could alleviate the over-fitting issues on few-shot learning application and increase the adaptability of prompts on unobserved instances. Specifically, modeling data-dependent prior enhances the adaptability of text features for both seen and unseen image features without the trade-off of performance between them. Based on the Bayesian framework, we utilize the Wasserstein gradient flow in the estimation of our target posterior distribution, which enables our prompt to be flexible in capturing the complex modes of image features. We demonstrate the effectiveness of our method on benchmark datasets for several experiments by showing statistically significant improvements on performance compared to existing methods.

show abstract

A Unified Particle-Optimization Framework for Scalable Bayesian Sampling

Cited by 22 publications

References 13 publications

Approximating the Optimal Transport Plan via Particle-Evolving Method

Approximating the Optimal Transport Plan via Particle-Evolving Method

Is MC Dropout Bayesian?

Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior

Contact Info

Product

Resources

About