A Logic for Inductive Probabilistic Reasoning

Abstract

Inductive probabilistic reasoning is understood as the application of inference patterns that use statistical background information to assign (subjective) probabilities to single events. The simplest such inference pattern is direct inference: from "70% of As are Bs" and "a is an A" infer that a is a B with probability 0.7. Direct inference is generalized by Jeffrey's rule and the principle of cross-entropy minimization. To adequately formalize inductive probabilistic reasoning is an interesting topic for artificial intelligence, as an autonomous system acting in a complex environment may have to base its actions on a probabilistic model of its environment, and the probabilities needed to form this model can often be obtained by combining statistical background information with particular observations made, i.e. by inductive probabilistic reasoning.In this paper a formal framework for inductive probabilistic reasoning is developed: syntactically it consists of an extension of the language of firstorder predicate logic that allows to express statements about both statistical and subjective probabilities. Semantics for this representation language are developed that give rise to two distinct entailment relations: a relation |= that models strict, probabilistically valid, inferences, and a relation |≈ that models inductive probabilistic inferences. The inductive entailment relation is obtained by implementing cross-entropy minimization in a preferred model semantics. A main objective of our approach is to ensure that for both entailment relations complete proof systems exist. This is achieved by allowing probability distributions in our semantic models that use non-standard probability values. A number of results are presented that show that in several important aspects the resulting logic behaves just like a logic based on realvalued probabilities alone.

On Learning to Prove

“…9. The use of probabilistic reasoning to model plausible reasoning is not a new idea-for instance, see work on probabilistic graphical models (Pearl, 1988) and work on inductive inference (e.g., see Solomonoff, 1964a,b;Jaeger, 2005). The field of automated reasoning (e.g., see Robinson and Voronkov, 2001b,a, for a survey) contains work on other forms of non-deductive reasoning including reasoning by induction (e.g., see Quinlan, 1986;Bundy, 2001;Comon, 2001), abduction (e.g., see Console et al, 1991;Mayer and Pirri, 1993;Gabbay et al, 1998;Denecker and Kakas, 2002), and analogy (e.g., see Davies and Russell, 1987;Ashley, 1988;Russell, 1988).…”

Coherent frequentism

“…11-12). This direct inference, reasoning from the frequency of individuals of a population that have a certain property to a level of certainty about whether a particular sample from the population, is a notable feature of inductive logic (e.g., Franklin, 2001;Jaeger, 2005) and often proves eective in everyday decisions. Knowing that the new cars of a certain model and year have speedometer readings within 1 mile per hour (mph) of the actual speed in 99.5% of cases, most drivers will, when betting on whether they comply with speed limits, have a high level of certainty that the speedometer readings of their particular new cars of that model and year accurately report their current speed in the absence of other relevant information.…”

“…95-96, 103-106), P x (ϑ ∈ Θ ) may be reported as an estimate of 1 Θ (θ) for use with currently unknown loss functions (cf. Jerey, 1986;Hwang 1992). That inferential role is currently played in many of the sciences by the p-value interpreted as a measure of evidence in signicance testing (Cox, 1977), but its notorious lack of coherence has prevented its universal acceptance (e.g., Royall, 1997).…”

“…While both approaches to coherence support the concept of placing bets in accord with the laws of probability, including conditional probability for called-o bets, none of the approaches entails the equality of conditional probability as dened by Kolmogorov and posterior probability as the hypothesis probability updated as a function of observed data. Replacing probabilities with proposition truth values and conditional probabilities with theorems (statements of implication) furnishes an illustration from deductive logic (Jerey, 1986): an agent whose set of propositions held to be true do not contradict each other at any point in time is completely selfconsistent. However, the agent cannot comply with the deductive version of the Bayesian temporal principle unless none of the truth values ever requires revision (Howson, 1997).…”

“…Thus, any decision-theoretic framework representing unknown values as random quantities mapped from some probability space stands on equal ground with Bayesianism as far as the minimal requirements of rationality are concerned. Such frameworks include geometric conditioning (Goldstein, 2001), probability kinematics (Diaconis and Zabell, 1982;Jerey, 2004), dynamic coherence (Skyrms, 1997;Zabell, 2002), and relative entropy maximization (Grünwald, 2004;Jaeger, 2005;Williamson, 2009) as well as condence decision theory (3.4).…”