Inductive Coherence

Garrabrant, Scott; Fallenstein, Benya; Demski, Abram; Soares, Nate

doi:10.48550/arxiv.1604.05288

Cited by 2 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Desideratum 3 dates back to at least Good [23], who proposes a weakening of the condition of coherence that could apply to the belief states of limited reasoners. Hacking [26] proposes an alternative weakening, as do Garrabrant et al [19].…”

Section: Desideratum 1 (Computable Approximability) the Methods For A...mentioning

confidence: 98%

A Formal Approach to the Problem of Logical Non-Omniscience

Garrabrant

Benson-Tilsen

Critch

et al. 2017

Electron. Proc. Theor. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

We present the logical induction criterion for computable algorithms that assign probabilities to every logical statement in a given formal language, and refine those probabilities over time. The criterion is motivated by a series of stock trading analogies. Roughly speaking, each logical sentence phi is associated with a stock that is worth $1 per share if phi is true and nothing otherwise, and we interpret the belief-state of a logically uncertain reasoner as a set of market prices, where pt_N(phi)=50% means that on day N, shares of phi may be bought or sold from the reasoner for 50%. A market is then called a logical inductor if (very roughly) there is no polynomial-time computable trading strategy with finite risk tolerance that earns unbounded profits in that market over time. We then describe how this single criterion implies a number of desirable properties of bounded reasoners; for example, logical inductors outpace their underlying deductive process, perform universal empirical induction given enough time to think, and place strong trust in their own reasoning process.Comment: In Proceedings TARK 2017, arXiv:1707.0825

show abstract

Section: Desideratum 1 (Computable Approximability) the Methods For A...mentioning

confidence: 98%

A Formal Approach to the Problem of Logical Non-Omniscience

Garrabrant

Benson-Tilsen

Critch

et al. 2017

Electron. Proc. Theor. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Desideratum 3 dates back to at least Good (1950), who proposes a weakening of the condition of coherence that could apply to the belief states of limited reasoners. Hacking (1967) proposes an alternative weakening, as do Garrabrant, Fallenstein, et al (2016).…”

Section: Desiderata For Reasoning Under Logical Uncertaintymentioning

confidence: 99%

“…6. Another notion of approximate coherence goes by the name of "inductive coherence" (Garrabrant, Fallenstein, et al 2016). A reasoner is called inductively coherent if (1) Pn(⊥) n 0; (2) Pn(φn) converges whenever φ is efficiently computable and each φn provably implies φn+1; and (3) for all efficiently computable sequences of provably mutually exclusive and exhaustive triplets (φn, ψn, χn), Pn(φn)+Pn(ψn)+Pn(χn) n 1.…”

Section: Affine Strengtheningsmentioning

confidence: 99%

Logical Induction

Garrabrant,

Benson-Tilsen,

Critch

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

We present a computable algorithm that assigns probabilities to every logical statement in a given formal language, and refines those probabilities over time. For instance, if the language is Peano arithmetic, it assigns probabilities to all arithmetical statements, including claims about the twin prime conjecture, the outputs of long-running computations, and its own probabilities. We show that our algorithm, an instance of what we call a logical inductor, satisfies a number of intuitive desiderata, including: (1) it learns to predict patterns of truth and falsehood in logical statements, often long before having the resources to evaluate the statements, so long as the patterns can be written down in polynomial time; (2) it learns to use appropriate statistical summaries to predict sequences of statements whose truth values appear pseudorandom; and (3) it learns to have accurate beliefs about its own current beliefs, in a manner that avoids the standard paradoxes of self-reference. For example, if a given computer program only ever produces outputs in a certain range, a logical inductor learns this fact in a timely manner; and if late digits in the decimal expansion of π are difficult to predict, then a logical inductor learns to assign ≈ 10% probability to "the nth digit of π is a 7" for large n. Logical inductors also learn to trust their future beliefs more than their current beliefs, and their beliefs are coherent in the limit (whenever φ → ψ, P∞(φ) ≤ P∞(ψ), and so on); and logical inductors strictly dominate the universal semimeasure in the limit.These properties and many others all follow from a single logical induction criterion, which is motivated by a series of stock trading analogies. Roughly speaking, each logical sentence φ is associated with a stock that is worth $1 per share if φ is true and nothing otherwise, and we interpret the belief-state of a logically uncertain reasoner as a set of market prices, where Pn(φ) = 50% means that on day n, shares of φ may be bought or sold from the reasoner for 50¢. The logical induction criterion says (very roughly) that there should not be any polynomial-time computable trading strategy with finite risk tolerance that earns unbounded profits in that market over time. This criterion bears strong resemblance to the "no Dutch book" criteria that support both expected utility theory (von Neumann and Morgenstern 1944) and Bayesian probability theory (Ramsey 1931;de Finetti 1937).

show abstract

Inductive Coherence

Cited by 2 publications

References 19 publications

A Formal Approach to the Problem of Logical Non-Omniscience

A Formal Approach to the Problem of Logical Non-Omniscience

Logical Induction

Contact Info

Product

Resources

About