We present the logical induction criterion for computable algorithms that
assign probabilities to every logical statement in a given formal language, and
refine those probabilities over time. The criterion is motivated by a series of
stock trading analogies. Roughly speaking, each logical sentence phi is
associated with a stock that is worth $1 per share if phi is true and nothing
otherwise, and we interpret the belief-state of a logically uncertain reasoner
as a set of market prices, where pt_N(phi)=50% means that on day N, shares of
phi may be bought or sold from the reasoner for 50%. A market is then called a
logical inductor if (very roughly) there is no polynomial-time computable
trading strategy with finite risk tolerance that earns unbounded profits in
that market over time. We then describe how this single criterion implies a
number of desirable properties of bounded reasoners; for example, logical
inductors outpace their underlying deductive process, perform universal
empirical induction given enough time to think, and place strong trust in their
own reasoning process.Comment: In Proceedings TARK 2017, arXiv:1707.0825
By considering agents to be a part of their environment, Orseau and Ring's space-time embedded intelligence [11] is a better fit to the real world than the traditional agent framework. However, a selfmodifying AGI that sees future versions of itself as an ordinary part of the environment may run into problems of self-reference. We show that in one particular model based on formal logic, naive approaches either lead to incorrect reasoning that allows an agent to put off an important task forever (the procrastination paradox), or fail to allow the agent to justify even obviously safe rewrites (the Löbian obstacle). We argue that these problems have relevance beyond our particular formalism, and discuss partial solutions.
Evidential Decision Theory (EDT) and Causal Decision Theory (CDT) are the leading contenders as theories of rational action, but both face counterexamples. We present some new counterexamples, including one in which the optimal action is causally dominated. We also present a novel decision theory, Functional Decision Theory (FDT), which simultaneously solves both sets of counterexamples. Instead of considering which physical action of theirs would give rise to the best outcomes, FDT agents consider which output of their decision function would give rise to the best outcome. This theory relies on a notion of subjunctive dependence, where multiple implementations of the same mathematical function are considered (even counterfactually) to have identical results for logical rather than causal reasons. Taking these subjunctive dependencies into account allows FDT agents to outperform CDT and EDT agents in, for example, the presence of accurate predictors.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.