A type theory for probability density functions

Bhat, Sooraj; Agarwal, Ashish; Vuduc, Richard; Gray, Alexander G.

doi:10.1145/2103656.2103721

Cited by 20 publications

(11 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the core calculus, the type of a primitive distribution characterizes the support of the distribution precisely: for a distribution 𝑑 of type dist(𝜏) and a value 𝑣, it holds that 𝑣 ∈ 𝑑.support if and only if 𝑣 is an inhabitant of type 𝜏. Primitive distributions can be generalized to density-carrying expressions [5,6] to further improve language expressibility.…”

Section: A Coroutine-based Pplmentioning

confidence: 99%

Sound probabilistic inference via guide types

Wang

Hoffmann

Reps³

2021

Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Probabilistic programming languages aim to describe and automate Bayesian modeling and inference. Modern languages support programmable inference, which allows users to customize inference algorithms by incorporating guide programs to improve inference performance. For Bayesian inference to be sound, guide programs must be compatible with model programs. One pervasive but challenging condition for model-guide compatibility is absolute continuity, which requires that the model and guide programs define probability distributions with the same support.This paper presents a new probabilistic programming language that guarantees absolute continuity, and features general programming constructs, such as branching and recursion. Model and guide programs are implemented as coroutines that communicate with each other to synchronize the set of random variables they sample during their execution. Novel guide types describe and enforce communication protocols between coroutines. If the model and guide are well-typed using the same protocol, then they are guaranteed to enjoy absolute continuity. An efficient algorithm infers guide types from code so that users do not have to specify the types. The new programming language is evaluated with an implementation that includes the type-inference algorithm and a prototype compiler that targets Pyro. Experiments show that our language is capable of expressing a variety of probabilistic models with nontrivial control flow and recursion, and that the coroutine-based computation does not introduce significant overhead in actual Bayesian inference.

show abstract

Section: A Coroutine-based Pplmentioning

confidence: 99%

Sound probabilistic inference via guide types

Wang

Hoffmann

Reps³

2021

Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

show abstract

“…For finite domains, a simple map can assign a probability to each possible value [30]. For continuous domains, one might reason about the density function algebraically [4]. For example, a Gaussian random variable with mean µ and variance σ 2 has density function…”

Section: Representing Distributionsmentioning

confidence: 99%

“…For example, if X = [4,6], then X/2 = [2,3]. Interval analysis is particularly suited to bounding floating point error in scientific computation.…”

Section: Domain-specific Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Uncertain< T >

Bornholt

Mytkowicz

McKinley

2014

Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

Emerging applications increasingly use estimates such as sensor data (GPS), probabilistic models, machine learning, big data, and human data. Unfortunately, representing this uncertain data with discrete types (floats, integers, and booleans) encourages developers to pretend it is not probabilistic, which causes three types of uncertainty bugs.(1) Using estimates as facts ignores random error in estimates.(2) Computation compounds that error. (3) Boolean questions on probabilistic data induce false positives and negatives.This paper introduces Uncertain T , a new programming language abstraction for uncertain data. We implement a Bayesian network semantics for computation and conditionals that improves program correctness. The runtime uses sampling and hypothesis tests to evaluate computation and conditionals lazily and efficiently. We illustrate with sensor and machine learning applications that Uncertain T improves expressiveness and accuracy.Whereas previous probabilistic programming languages focus on experts, Uncertain T serves a wide range of developers. Experts still identify error distributions. However, both experts and application writers compute with distributions, improve estimates with domain knowledge, and ask questions with conditionals. The Uncertain T type system and operators encourage developers to expose and reason about uncertainty explicitly, controlling false positives and false negatives. These benefits make Uncertain T a compelling programming model for modern applications facing the challenge of uncertainty. a A Figure 1. The probability distribution A quantifies potential errors. Sampling A produces a single point a, introducing uncertainty, but a does not necessarily represent the true value. Many programs treat a as the true value.

show abstract

“…This can be checked using static analyses (Bhat et al 2012a) that can determine if M [[O[[M]]]] is absolutely continuous and thus admits a density. A more sensitive analysis could additionally determine the discontinuities of the density function in many cases, allowing to check compatibility statically.…”

Section: Monadic Semantics For Fun With Failmentioning

confidence: 99%

A model-learner pattern for bayesian reasoning

Gordon

Aizatulin

Borgström

et al. 2013

Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

View full text Add to dashboard Cite

A Bayesian model is based on a pair of probability distributions, known as the prior and sampling distributions. A wide range of fundamental machine learning tasks, including regression, classification, clustering, and many others, can all be seen as Bayesian models. We propose a new probabilistic programming abstraction, a typed Bayesian model, based on a pair of probabilistic expressions for the prior and sampling distributions. A sampler for a model is an algorithm to compute synthetic data from its sampling distribution, while a learner for a model is an algorithm for probabilistic inference on the model. Models, samplers, and learners form a generic programming pattern for model-based inference. They support the uniform expression of common tasks including model testing, and generic compositions such as mixture models, evidencebased model averaging, and mixtures of experts. A formal semantics supports reasoning about model equivalence and implementation correctness. By developing a series of examples and three learner implementations based on exact inference, factor graphs, and Markov chain Monte Carlo, we demonstrate the broad applicability of this new programming pattern.

show abstract

A type theory for probability density functions

Cited by 20 publications

References 14 publications

Sound probabilistic inference via guide types

Sound probabilistic inference via guide types

Uncertain< T >

A model-learner pattern for bayesian reasoning

Contact Info

Product

Resources

About