Probabilistic Programming Inference via Intensional Semantics

Castellan, Simon; Paquet, Hugo

doi:10.1007/978-3-030-17184-1_12

Cited by 10 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Trace-based [Borgström et al 2016;Castellan and Paquet 2019]: The state-to-state transition relation is interpreted under a fixed trace of random sources (e.g., a trace of coin flips), where the random constructs (e.g., coin flips) are resolved as deterministic readouts from the trace. In our setting, however, this approach lacks a mechanism for tracking the correlations among traces, which have been shown to be important for expected-cost analysis [Wang et al 2020b].…”

Section: Difficulties For Standard Approachesmentioning

confidence: 99%

Work Analysis with Resource-Aware Session Types

Das

Hoffmann

Pfenning

2018

Proceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science

View full text Add to dashboard Cite

While there exist several successful techniques for supporting programmers in deriving static resource bounds for sequential code, analyzing the resource usage of message-passing concurrent processes poses additional challenges. To meet these challenges, this article presents an analysis for statically deriving worst-case bounds on the total work performed by message-passing processes. To decompose interacting processes into components that can be analyzed in isolation, the analysis is based on novel resource-aware session types, which describe protocols and resource contracts for inter-process communication. A key innovation is that both messages and processes carry potential to share and amortize cost while communicating. To symbolically express resource usage in a setting without static data structures and intrinsic sizes, resource contracts describe bounds that are functions of interactions between processes. Resource-aware session types combine standard binary session types and type-based amortized resource analysis in a linear type system. This type system is formulated for a core session-type calculus of the language SILL and proved sound with respect to a multiset-based operational cost semantics that tracks the total number of messages that are exchanged in a system. The effectiveness of the analysis is demonstrated by analyzing standard examples from amortized analysis and the literature on session types and by a comparative performance analysis of different concurrent programs implementing the same interface. arXiv:1712.08310v2 [cs.PL] 27 Apr 2018components that can be analyzed in isolation. After all, the resource usage of each component crucially depends on its interactions with the world.In this paper, we study the foundations of worst-case resource analysis for message-passing processes. A key idea of our approach is to rely on resourceaware session types to describe structure, protocols, and resource bounds for inter-process communication that we can use to perform a compositional and precise amortized analysis. Session types [32,33,12,13,45] prescribe bidirectional communication protocols for message-passing processes. Binary session types govern the interaction of two processes along a single channel, prescribing complementary send and receive actions for the processes at the two endpoints of a channel. We use such protocols as the basis of resource usage contracts that not only specify the type but also the potential of a message that is sent along a channel. The potential (in the sense of classic amortized analysis [43]) may be spent sending other messages as part of the network of interacting processes, or maintained locally for future interactions. Resource analysis is static, using the type system, and the runtime behavior of programs is not affected.

show abstract

Section: Difficulties For Standard Approachesmentioning

confidence: 99%

Work Analysis with Resource-Aware Session Types

Das

Hoffmann

Pfenning

2018

Proceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science

View full text Add to dashboard Cite

show abstract

“…Both operational and denotational models have recently been applied to the validation of inference algorithms: see e.g. [20,8] for the former and [45,10] for the latter. There are other approaches, e.g.…”

Section: Introductionmentioning

confidence: 99%

Densities of Almost Surely Terminating Probabilistic Programs are Differentiable Almost Everywhere

Mak

Ong

Paquet

et al. 2021

Programming Languages and Systems

Self Cite

View full text Add to dashboard Cite

We study the differential properties of higher-order statistical probabilistic programs with recursion and conditioning. Our starting point is an open problem posed by Hongseok Yang: what class of statistical probabilistic programs have densities that are differentiable almost everywhere? To formalise the problem, we consider Statistical PCF (SPCF), an extension of call-by-value PCF with real numbers, and constructs for sampling and conditioning. We give SPCF a sampling-style operational semantics à la Borgström et al., and study the associated weight (commonly referred to as the density) function and value function on the set of possible execution traces.Our main result is that almost surely terminating SPCF programs, generated from a set of primitive functions (e.g. the set of analytic functions) satisfying mild closure properties, have weight and value functions that are almost everywhere differentiable. We use a stochastic form of symbolic execution to reason about almost everywhere differentiability. A by-product of this work is that almost surely terminating deterministic (S)PCF programs with real parameters denote functions that are almost everywhere differentiable.Our result is of practical interest, as almost everywhere differentiability of the density function is required to hold for the correctness of major gradient-based inference algorithms.

show abstract

“…It should be clear that probabilistic languages with sampling instructions can be interpreted in this model. We expect strong connections with game semantics models dealing with such languages [3], although our model differs from the start by its intention. In particular, types arise from the behaviour of processes and are therefore not predefined.…”

Section: Discussionmentioning

confidence: 99%

Zeta Functions and the (Linear) Logic of Markov Processes

Seiller

2020

Preprint

View full text Add to dashboard Cite

In a series of papers, the author introduced models of linear logic known as "Interaction Graphs". These models generalise Girard's various geometry of interaction constructions, providing a unifying framework for those. In this work, we exhibit how these models can be understood mathematically through a cocycle property satisfied by zeta functions of dynamical systems. Focussing on probabilistic models, we then explain how the notion of graphings used in the models captures a natural class of Markov processes. We further extend previous constructions to provide a model of linear logic as a type system over the set of all (discrete-time, time-independent) Markov processes.

show abstract

Probabilistic Programming Inference via Intensional Semantics

Cited by 10 publications

References 14 publications

Work Analysis with Resource-Aware Session Types

Work Analysis with Resource-Aware Session Types

Densities of Almost Surely Terminating Probabilistic Programs are Differentiable Almost Everywhere

Zeta Functions and the (Linear) Logic of Markov Processes

Contact Info

Product

Resources

About