A note on the inevitability of maximum entropy

Paris, Jeffrey; Vencovská, Alena

doi:10.1016/0888-613x(90)90020-3

Cited by 116 publications

(71 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…What they are not is estimates of objective probabilities. emphasis original then Williamson's account (Maxent in Paris's terms) seems to be the clear objective Bayesian frontrunner, see [21,22,24].…”

Section: The Right Inference Process?mentioning

confidence: 99%

Invariant Equivocation

Landes

Masterton

2016

Erkenn

View full text Add to dashboard Cite

Objective Bayesians hold that degrees of belief ought to be chosen in the set of probability functions calibrated with one's evidence. The particular choice of degrees of belief is via some objective, i.e., not agentdependent, inference process that, in general, selects the most equivocal probabilities from among those compatible with one's evidence. Maximising entropy is what drives these inference processes in recent works by Williamson and Masterton though they disagree as to what should have its entropy maximised. With regard to the probability function one should adopt as one's belief function, Williamson advocates selecting the probability function with greatest entropy compatible with one's evidence while Masterton advocates selecting the expected probability function relative to the density function with greatest entropy compatible with one's evidence. In this paper we discuss the significant relative strengths of these two positions. In particular, Masterton's original proposal is further developed and investigated to reveal its significant properties; including its equivalence to the centre of mass inference process and its ability to accommodate higher order evidence.Keywords: Maximum Entropy, Objective Bayesianism, Inference Process, Centre of Mass, Language Invariance 1 Introduction "How should one form graded beliefs?" is a question that has long fascinated philosophers. The answer to this question is highly relevant throughout science, law, operational research and policy-making. Intuitively, it is obvious that one's evidence ought to matter when forming beliefs, whether graded or binary. The best way of caching out this ubiquitous intuition is, however, a matter of lively disagreement. The main philosophical protagonists in this debate are: the subjective Bayesians, who can be further subdivided into the radical (de Finetti This paper is concerned with objective Bayesianism. Objective Bayesians, like their subjective brethren, hold that one's degrees of belief ought to be consistent with one's evidence. However, in cases where the evidential constraints are satisfied by more than one probability function the objective Bayesians differ from the subjective Bayesians in holding that the choice of a particular probability function from those consistent with the evidential constraints ought to made 1 in an objective-i.e., agent-independent-manner. Typically, such a choice is made by applying an inference process that picks from among the probability functions consistent with the evidential constraints that function which is, in some sense, maximally equivocal. 1 Objective Bayesianism is (still!) a minority view with a relatively small but dedicated group of advocates. Even in this relatively small group there are disagreements. One such recent disagreement is that between Williamson [27] and Masterton [16] on how to understand maximal equivocation.We shall herein first rehearse Williamson's account in Section 2.1 and 2.2. Then we improve on Masterton's approach by reformulating it, Section 2...

show abstract

Section: The Right Inference Process?mentioning

confidence: 99%

Invariant Equivocation

Landes

Masterton

2016

Erkenn

View full text Add to dashboard Cite

show abstract

“…The following two theorems state two key structural properties of cross-entropy minimization. These properties are the cornerstones of Shore's and Johnson's (1980) axiomatic justification of cross-entropy minimization, and, in a somewhat different guise, also of Paris's and Vencovská's (1990) derivation of the maximum entropy principle.…”

Section: A1 Qualitative Propertiesmentioning

confidence: 99%

A Logic for Inductive Probabilistic Reasoning

Jaeger

2005

Uncertainty, Rationality, and Agency

View full text Add to dashboard Cite

Inductive probabilistic reasoning is understood as the application of inference patterns that use statistical background information to assign (subjective) probabilities to single events. The simplest such inference pattern is direct inference: from "70% of As are Bs" and "a is an A" infer that a is a B with probability 0.7. Direct inference is generalized by Jeffrey's rule and the principle of cross-entropy minimization. To adequately formalize inductive probabilistic reasoning is an interesting topic for artificial intelligence, as an autonomous system acting in a complex environment may have to base its actions on a probabilistic model of its environment, and the probabilities needed to form this model can often be obtained by combining statistical background information with particular observations made, i.e. by inductive probabilistic reasoning.In this paper a formal framework for inductive probabilistic reasoning is developed: syntactically it consists of an extension of the language of firstorder predicate logic that allows to express statements about both statistical and subjective probabilities. Semantics for this representation language are developed that give rise to two distinct entailment relations: a relation |= that models strict, probabilistically valid, inferences, and a relation |≈ that models inductive probabilistic inferences. The inductive entailment relation is obtained by implementing cross-entropy minimization in a preferred model semantics. A main objective of our approach is to ensure that for both entailment relations complete proof systems exist. This is achieved by allowing probability distributions in our semantic models that use non-standard probability values. A number of results are presented that show that in several important aspects the resulting logic behaves just like a logic based on realvalued probabilities alone.

show abstract

“…What is irrational is the behaviour of a man who would persist in subscribing to an indefensible statement after its indefensibility has been made know to him (Hintikka, 1962, p.109) The characterisation of the Maximum Entropy inference process as the unique choice of a probability distribution consistent with an agent's knowledge and with the common sense principles outlined above was first given in Paris and Vencovská (1990), and is fully developed in chapter 7 of Paris (1994), culminating in Theorem 7.9. See also Paris (1999) for the unification of the common sense principles under the "Symmetry Principle", and Paris and Vencovská (2001) for the study of the non linear case.…”

Section: Further Commentsmentioning

confidence: 99%