PAC-Bayesian model averaging

McAllester, David

doi:10.1145/307400.307435

Cited by 236 publications

(219 citation statements)

References 9 publications

Supporting

Mentioning

215

Contrasting

Order By: Relevance

“…The appealing aspect of integrating features of PAC-learning with Bayesian inference is that PAC-bounds are robust even when the true hypothesis is not included in the hypothesis space. However, known PAC-Bayesian results only imply that probability matching on the posterior is optimal (i.e., provides the tightest PAC-Bayesian bounds) for tasks that already contain in their description some aspect of probability matching, such as estimating a distribution, selecting a hypothesis stochastically, or providing a weighted average (McAllester, 1999(McAllester, , 2003. In fact, for the task of interest to us⎯selecting the true hypothesis⎯PAC-Bayesian considerations imply the same response as rational choice:…”

Section: Changing the Circumstancesmentioning

confidence: 99%

Confirmation in the Cognitive Sciences: The Problematic Case of Bayesian Models

Eberhardt

Danks

2011

Minds & Machines

View full text Add to dashboard Cite

Abstract:Bayesian models of human learning are becoming increasingly popular in cognitive science. We argue that their purported confirmation largely relies on a methodology that depends on premises that are inconsistent with the claim that people are Bayesian about learning and inference. Bayesian models in cognitive science derive their appeal from their normative claim that the modeled inference is in some sense rational. Standard accounts of the rationality of Bayesian inference imply predictions that an agent selects the option that maximizes the posterior expected utility. Experimental confirmation of the models, however, has been claimed because of groups of agents that "probability match" the posterior. Probability matching only constitutes support for the Bayesian claim if additional unobvious and untested (but testable) assumptions are invoked. The alternative strategy of weakening the underlying notion of rationality no longer distinguishes the Bayesian model uniquely. A new account of rationality-either for inference or for decision-making-is required to successfully confirm Bayesian models in cognitive science.

show abstract

Section: Changing the Circumstancesmentioning

confidence: 99%

Confirmation in the Cognitive Sciences: The Problematic Case of Bayesian Models

Eberhardt

Danks

2011

Minds & Machines

View full text Add to dashboard Cite

show abstract

“…the generalization error. Well known examples of this type of bound are the Vapnik-Chervonenkis bounds (VC bounds) [39], Probably Approximately Correct Bayes bounds (PAC Bayes bounds) [23], Occam's Razor bounds [7], Sample Compression bounds [11] and Rademacher Complexity bounds [4]. The author infers from the comparison of these two categories that test set bounds are generally much tighter than training set bounds and are a superior tool in reporting error rates.…”

Section: Preliminaries and Motivationmentioning

confidence: 99%

“…In [3] improved PAC Bayes bounds are provided for a class of linear classifiers. These bounds are tighter than the ones previously introduced in [23]. In [6] bounds are provided for a validation technique called progressive validation which are tighter than those for hold-out-set validation.…”

Section: Related Workmentioning

confidence: 99%

“…Distribution free bounds relate these two errors by providing us with probabilistic estimates for the generalization error given the empirical error without knowledge of the underlying joint distribution. This is the primary reason for deriving such bounds [16,39,10,6,23,22]. A common characteristic of all of these bounds is that they assume the data is i.i.d.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Distribution-free bounds for relational classification

Dhurandhar

Dobra

2011

Knowl Inf Syst

View full text Add to dashboard Cite

Statistical Relational Learning (SRL) is a sub-area in Machine Learning which addresses the problem of performing statistical inference on data that is correlated and not independently and identically distributed (i.i.d.) -as is generally assumed. For the traditional i.i.d. setting, distribution free bounds exist, such as the Hoeffding bound, which are used to provide confidence bounds on the generalization error of a classification algorithm given its hold-out error on a sample size of N . Bounds of this form are currently not present for the type of interactions that are considered in the data by relational classification algorithms. In this paper we extend the Hoeffding bounds to the relational setting. In particular, we derive distribution free bounds for certain classes of data generation models that do not produce i.i.d. data and are based on the type of interactions that are considered by relational classification algorithms that have been developed in SRL. We conduct empirical studies on synthetic and real data which show that these data generation models are indeed realistic and the derived bounds are tight enough for practical use.

show abstract

“…The algorithm stated here is in a suboptimal form, which is good enough for our purposes (see [McAllester, 1999] for more sophisticated versions): is achieved for at least one c ∈ C = {c 0 , c 1 , . .…”

Section: A Consistent Algorithm: Proof Of Theoremmentioning

confidence: 99%

Suboptimal Behavior of Bayes and MDL in Classification Under Misspecification

Grünwald

Langford

2004

Learning Theory

View full text Add to dashboard Cite

Abstract. We show that forms of Bayesian and MDL inference that are often applied to classification problems can be inconsistent. This means that there exists a learning problem such that for all amounts of data the generalization errors of the MDL classifier and the Bayes classifier relative to the Bayesian posterior both remain bounded away from the smallest achievable generalization error. From a Bayesian point of view, the result can be reinterpreted as saying that Bayesian inference can be inconsistent under misspecification, even for countably infinite models. We extensively discuss the result from both a Bayesian and an MDL perspective.

show abstract

PAC-Bayesian model averaging

Cited by 236 publications

References 9 publications

Confirmation in the Cognitive Sciences: The Problematic Case of Bayesian Models

Confirmation in the Cognitive Sciences: The Problematic Case of Bayesian Models

Distribution-free bounds for relational classification

Suboptimal Behavior of Bayes and MDL in Classification Under Misspecification

Contact Info

Product

Resources

About