Compiling Stan to generative probabilistic languages and extension to deep probabilistic programming

Baudart, Guillaume; Burroni, Javier; Hirzel, Martin; Mandel, Louis; Shinnar, Avraham

doi:10.1145/3453483.3454058

Cited by 6 publications

(3 citation statements)

References 18 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The goal is to pay a large one-time cost in training a "general" inference model which may not be adequate for inference in all settings, but can be quickly adapted to new datasets with low cost. To demonstrate the foundation posterior, we meta-amortize inference over a set of standard Stan [12] programs from PosteriorDB [42], a benchmark dataset for evaluating inference algorithms [4,5,65,66,20].…”

Section: Foundation Posteriormentioning

confidence: 99%

Foundation Posteriors for Approximate Probabilistic Inference

Wu¹,

Goodman²

2022

Preprint

View full text Add to dashboard Cite

Probabilistic programs provide an expressive representation language for generative models. Given a probabilistic program, we are interested in the task of posterior inference: estimating a latent variable given a set of observed variables. Existing techniques for inference in probabilistic programs often require choosing many hyper-parameters, are computationally expensive, and/or only work for restricted classes of programs. Here we formulate inference as masked language modeling: given a program, we generate a supervised dataset of variables and assignments, and randomly mask a subset of the assignments. We then train a neural network to unmask the random values, defining an approximate posterior distribution. By optimizing a single neural network across a range of programs we amortize the cost of training, yielding a "foundation" posterior able to do zero-shot inference for new programs. The foundation posterior can also be fine-tuned for a particular program and dataset by optimizing a variational inference objective. We show the efficacy of the approach, zero-shot and fine-tuned, on a benchmark of STAN programs.Preprint. Under review.

show abstract

Section: Foundation Posteriormentioning

confidence: 99%

Foundation Posteriors for Approximate Probabilistic Inference

Wu¹,

Goodman²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Users can now try a range of synthesized guides on a given model before attempting to craft their own. We recently proposed new backends for the Stanc3 Compiler to Pyro and NumPyro 1 and showed how to extend Stan with support for explicit variational guides [Baudart et al 2021]. In this paper, we show that our compiler and runtime can be used to test NumPyro autoguides on Stan models, and evaluate our approach on PosteriorDB a database of Stan models with corresponding data, and reference posterior samples [Vehtari and Magnusson 2020].…”

Section: Motivationmentioning

confidence: 99%

Automatic Guide Generation for Stan via NumPyro

Baudart¹,

Mandel²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Stan is a very popular probabilistic language with a state-of-the-art HMC sampler but it only offers a limited choice of algorithms for black-box variational inference. In this paper, we show that using our recently proposed compiler from Stan to Pyro, Stan users can easily try the set of algorithms implemented in Pyro for black-box variational inference. We evaluate our approach on PosteriorDB, a database of Stan models with corresponding data and reference posterior samples. Results show that the eight algorithms available in Pyro offer a range of possible compromises between complexity and accuracy. This paper illustrates that compiling Stan to another probabilistic language can be used to leverage new features for Stan users, and give access to a large set of examples for language developers who implement these new features.

show abstract

“…Baudart et al (2019) describe a new probabilistic programming language, Yaps, which is inspired by SlicStan to allow for a concise Python-based frontend for Stan. In addition, the density-based semantics of Stan that the paper presents has been used as a basis to formalise a procedure for translating Stan to generative (implicit) PPLs (Baudart et al, 2021). AQUA (Huang et al, 2021) is a PPL that uses symbolic inference and quantization of the probability density, whose semantics has also been inspired by that of SlicStan.…”

Section: Impactmentioning

confidence: 99%

Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic

Gorinova

Gordon

Sutton

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Stan is a probabilistic programming language that has been increasingly used for real-world scalable projects. However, to make practical inference possible, the language sacrifices some of its usability by adopting a block syntax, which lacks compositionality and flexible user-defined functions. Moreover, the semantics of the language has been mainly given in terms of intuition about implementation, and has not been formalised.This paper provides a formal treatment of the Stan language, and introduces the probabilistic programming language SlicStan -a compositional, self-optimising version of Stan. Our main contributions are (1) the formalisation of a core subset of Stan through an operational density-based semantics; (2) the design and semantics of the Stan-like language SlicStan, which facilities better code reuse and abstraction through its compositional syntax, more flexible functions, and information-flow type system; and (3) a formal, semanticpreserving procedure for translating SlicStan to Stan.

show abstract

Compiling Stan to generative probabilistic languages and extension to deep probabilistic programming

Cited by 6 publications

References 18 publications

Foundation Posteriors for Approximate Probabilistic Inference

Foundation Posteriors for Approximate Probabilistic Inference

Automatic Guide Generation for Stan via NumPyro

Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic

Contact Info

Product

Resources

About