Some PAC-Bayesian theorems

McAllester, David

doi:10.1145/279943.279989

Cited by 269 publications

(317 citation statements)

References 7 publications

Supporting

Mentioning

313

Contrasting

Order By: Relevance

“…Lemma 1 (McAllester, 1998). For any probability distribution P on a countable rule class C we have the following.…”

Section: Summary Of the Main Resultsmentioning

confidence: 99%

“…A PAC-Bayesian approach to machine learning attempts to combine the advantages of both PAC and Bayesian approaches (Shawe-Taylor & Williamson, 1997;McAllester, 1998). The Bayesian approach has the advantage of using arbitrary domain knowledge in the form of a Bayesian prior.…”

Section: Introductionmentioning

confidence: 99%

“…Perhaps the simplest example of a PAC-Bayesian theorem is noted in McAllester (1998). Consider a countable class of concepts f 1 , f 2 , f 3 , .…”

Section: Introductionmentioning

confidence: 99%

“…Let S be a sample of m pairs drawn independently according to D and defineˆ ( f i ) to be the fraction of pairs x, y in S for which f i (x) = y. Hereˆ ( f i ) is a measure of how well f i fits the training data and log 1 P( f i ) can be viewed as the "description length" of the concept f i . It is noted in McAllester (1998) that a simple combination of Chernoff and union bounds yields that with probability at least 1 − δ over the choice of the sample S we have the following for all f i .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Untitled

McAllester

2003

Machine Learning

Self Cite

View full text Add to dashboard Cite

Abstract. PAC-Bayesian learning methods combine the informative priors of Bayesian methods with distribution-free PAC guarantees. Stochastic model selection predicts a class label by stochastically sampling a classifier according to a "posterior distribution" on classifiers. This paper gives a PAC-Bayesian performance guarantee for stochastic model selection that is superior to analogous guarantees for deterministic model selection. The guarantee is stated in terms of the training error of the stochastic classifier and the KL-divergence of the posterior from the prior. It is shown that the posterior optimizing the performance guarantee is a Gibbs distribution. Simpler posterior distributions are also derived that have nearly optimal performance guarantees.

show abstract

“…Lemma 1 (McAllester, 1998). For any probability distribution P on a countable rule class C we have the following.…”

Section: Summary Of the Main Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Perhaps the simplest example of a PAC-Bayesian theorem is noted in McAllester (1998). Consider a countable class of concepts f 1 , f 2 , f 3 , .…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Untitled

McAllester

2003

Machine Learning

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using PAC-Bayesian theory [41,33,15,16,18,43], we derive oracle inequalities for the pseudo-posterior mean. We obtain rates of convergence for these estimators in the complete measurement setting.…”

Section: Introductionmentioning

confidence: 99%

Pseudo-Bayesian quantum tomography with rank-adaptation

Mai

Alquier

2017

Journal of Statistical Planning and Inference

View full text Add to dashboard Cite

Quantum state tomography, an important task in quantum information processing, aims at reconstructing a state from prepared measurement data. Bayesian methods are recognized to be one of the good and reliable choices in estimating quantum states [9]. Several numerical works showed that Bayesian estimations are comparable to, and even better than other methods in the problem of 1-qubit state recovery. However, the problem of choosing prior distribution in the general case of n qubits is not straightforward. More importantly, the statistical performance of Bayesian type estimators have not been studied from a theoretical perspective yet. In this paper, we propose a novel prior for quantum states (density matrices), and we define pseudo-Bayesian estimators of the density matrix. Then, using PAC-Bayesian theorems [16], we derive rates of convergence for the posterior mean. The numerical performance of these estimators are tested on simulated and real datasets.

show abstract

Generalisation bounds for kernel PCA through PAC‐Bayes learning

Haddouche,

Guedj,

Shawe‐Taylor

2024

Stat

View full text Add to dashboard Cite

Principal component analysis(PCA) is a popular method for dimension reduction and has attracted an unfailing interest for decades. More recently, kernel PCA (KPCA) has emerged as an extension of PCA, but despite its use in practice, a sound theoretical understanding of KPCA is missing. We contribute several empirical generalisation bounds on the efficiency of KPCA, involving the empirical eigenvalues of the kernel Gram matrix. Our bounds are derived through the use of probably approximately correct (PAC)‐Bayes theory and highlight the importance of some desirable properties of datasets, expressed as variance‐typed terms, to attain fast rates, achievable for a wide class of kernels.

show abstract

Some PAC-Bayesian theorems

Cited by 269 publications

References 7 publications

Untitled

Untitled

Pseudo-Bayesian quantum tomography with rank-adaptation

Generalisation bounds for kernel PCA through PAC‐Bayes learning

Contact Info

Product

Resources

About