Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic

Abstract

Stan is a probabilistic programming language that has been increasingly used for real-world scalable projects. However, to make practical inference possible, the language sacrifices some of its usability by adopting a block syntax, which lacks compositionality and flexible user-defined functions. Moreover, the semantics of the language has been mainly given in terms of intuition about implementation, and has not been formalised.This paper provides a formal treatment of the Stan language, and introduces the probabilistic programming language SlicStan -a compositional, self-optimising version of Stan. Our main contributions are (1) the formalisation of a core subset of Stan through an operational density-based semantics; (2) the design and semantics of the Stan-like language SlicStan, which facilities better code reuse and abstraction through its compositional syntax, more flexible functions, and information-flow type system; and (3) a formal, semanticpreserving procedure for translating SlicStan to Stan.

Compiling Stan to generative probabilistic languages and extension to deep probabilistic programming

“…In Stan, a model represents the unnormalized density of the joint distribution of the parameters defined in the parameters block given the data defined in the data block [7,15]. A Stan program can thus be viewed as a function from parameters Table 1.…”

“…A Stan model can be described using classic imperative statements, plus two special statements that modify the value of target. The first one, target+= e, increments the value of target by e. The second one, e~D, is equivalent to target+= D lpdf (e) [15] where D lpdf denotes the log probability density function of D.…”

“…While the semantics of probabilistic languages have been extensively studied [14,15,18,30], to the best of our knowledge little is known about the relationship between Stan and generative PPLs. We show that a simple 1:1 translation is incorrect or incomplete for a set of subtle but widely-used Stan features, such as left expressions or implicit priors.…”

Deployable probabilistic programming

“…Stan [10] is built around an imperative probabilistic programming language with program structure tuned for most efficient execution of inference on the model. SlicStan [17] is built on top of Stan as a sourceto-source compiler, providing the model developer with a more intuitive language while retaining performance benefits of Stan. Our work differs from existing approaches in that we advocate enabling probabilistic programming in a general-purpose programming language by leveraging existing capabilities of the language, rather than building a new language on top or besides an existing one.…”

“…(lines [12][13][14][15][16][17]. If only the maximum a posteriori is estimated, the parameter values will maximize their likelihood given the observations:…”