Probabilistic Inference by Program Transformation in Hakaru (System Description)

We propose trace abstraction modulo probability, a proof technique for verifying high-probability accuracy guarantees of probabilistic programs. Our proofs overapproximate the set of program traces using failure automata, nite-state automata that upper bound the probability of failing to satisfy a target speci cation.We automate proof construction by reducing probabilistic reasoning to logical reasoning: we use program synthesis methods to select axioms for sampling instructions, and then apply Craig interpolation to prove that traces fail the target speci cation with only a small probability. Our method handles programs with unknown inputs, parameterized distributions, in nite state spaces, and parameterized speci cations. We evaluate our technique on a range of randomized algorithms drawn from the di erential privacy literature and beyond. To our knowledge, our approach is the rst to automatically establish accuracy properties of these algorithms.common weaknesses: they are mostly restricted to closed programs with xed inputs and nite state spaces, and support for properties with symbolic parameters remains limited.In this paper, we start from established automated veri cation techniques for non-probabilistic programs and extend them to the probabilistic se ing. Our logic-based approach yields several bene ts. By reasoning symbolically instead of numerically, we can (i) directly establish properties for all inputs rather than requiring xed inputs, (ii) handle programs that sample from distributions with unknown parameters, possibly over in nite ranges, and (iii) prove parametric accuracy properties, making it possible to automatically establish tradeo s between accuracy and failure probabilities, and capture the dependence on other input parameters.

Section: Related Workmentioning

confidence: 99%

Trace abstraction modulo probability

Smith

Hsu

Albarghouthi

2019

“…Other languages and systems include Hakaru [Narayanan et al 2016], Figaro [Pfeffer 2009], Fun [Borgström et al 2011], Greta [Golding et al 2018] and many others.…”

Section: Related Workmentioning

confidence: 99%

“…Claret et al [2013] present a new inference algorithm that is based on data-flow analysis. Hakaru [Narayanan et al 2016] is a relatively new probabilistic programming language embedded in Haskell, which performs automatic and semantic-preserving transformations on the program, in order to calculate conditional distributions and perform exact inference by computer algebra. The PSI system [Gehr et al 2016] analyses probabilistic programs using a symbolic domain, and outputs a simplified expression representing the posterior distribution.…”

Section: Static Analysis For Probabilistic Programming Languagesmentioning

confidence: 99%

Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic

Gorinova

Gordon

Sutton

2019

Stan is a probabilistic programming language that has been increasingly used for real-world scalable projects. However, to make practical inference possible, the language sacrifices some of its usability by adopting a block syntax, which lacks compositionality and flexible user-defined functions. Moreover, the semantics of the language has been mainly given in terms of intuition about implementation, and has not been formalised.This paper provides a formal treatment of the Stan language, and introduces the probabilistic programming language SlicStan -a compositional, self-optimising version of Stan. Our main contributions are (1) the formalisation of a core subset of Stan through an operational density-based semantics; (2) the design and semantics of the Stan-like language SlicStan, which facilities better code reuse and abstraction through its compositional syntax, more flexible functions, and information-flow type system; and (3) a formal, semanticpreserving procedure for translating SlicStan to Stan.

“…This property says there is no implicit sequential state in the language. It is essential for Shan and Ramsey's disintegration-based exact Bayesian inference technique [2017], implemented in the Hakaru system [Narayanan et al 2016]. The corollary is related to Fubini's theorem for reordering integrals: informally, d d ( , ) = d d ( , ).…”

Section: Introductionmentioning

confidence: 99%

A domain theory for statistical probabilistic programming

Vákár

Kammar

Staton

2019

We give an adequate denotational semantics for languages with recursive higher-order types, continuous probability distributions, and soft constraints. These are expressive languages for building Bayesian models of the kinds used in computational statistics and machine learning. Among them are untyped languages, similar to Church and WebPPL, because our semantics allows recursive mixed-variance datatypes. Our semantics justifies important program equivalences including commutativity.Our new semantic model is based on 'quasi-Borel predomains'. These are a mixture of chain-complete partial orders (cpos) and quasi-Borel spaces. Quasi-Borel spaces are a recent model of probability theory that focuses on sets of admissible random elements. Probability is traditionally treated in cpo models using probabilistic powerdomains, but these are not known to be commutative on any class of cpos with higher order functions. By contrast, quasi-Borel predomains do support both a commutative probabilistic powerdomain and higher-order functions. As we show, quasi-Borel predomains form both a model of Fiore's axiomatic domain theory and a model of Kock's synthetic measure theory.