Bayesian inversion is at the heart of probabilistic programming and more generally machine learning. Understanding inversion is made difficult by the pointful (kernel-centric) point of view usually taken in the literature. We develop a pointless (kernel-free) approach to inversion. While doing so, we revisit some foundational objects of probability theory, unravel their category-theoretical underpinnings and show how pointless Bayesian inversion sits naturally at the centre of this construction. P (d) · P (h | d) = P (d | h) · P (h)(1)
This paper introduces a categorical framework to study the exact and approximate semantics of probabilistic programs. We construct a dagger symmetric monoidal category of Borel kernels where the dagger-structure is given by Bayesian inversion. We show functorial bridges between this category and categories of Banach lattices which formalize the move from kernel-based semantics to predicate transformer (backward) or state transformer (forward) semantics. These bridges are related by natural transformations, and we show in particular that the Radon-Nikodym and Riesz representation theorems -two pillars of probability theory -define natural transformations.With the mathematical infrastructure in place, we present a generic and endogenous approach to approximating kernels on standard Borel spaces which exploits the involutive structure of our category of kernels. The approximation can be formulated in several equivalent ways by using the functorial bridges and natural transformations described above. Finally, we show that for sensible discretization schemes, every Borel kernel can be approximated by kernels on finite spaces, and that these approximations converge for a natural choice of topology.We illustrate the theory by showing two examples of how approximation can effectively be used in practice: Bayesian inference and the Kleene * operation of ProbNetKAT. . We introduce the category BL σ of Banach lattices and σorder continuous positive operators as well as the Köthe dual functor (−) σ : BL op σ → BL σ ( §3). These will play a central role in studying convergence of our approximation schemes. 3. We provide the first 2 categorical understanding of the Radon-Nikodym and the Riesz representation theorems. These arise as natural transformations between two functors relating kernels and Banach lattices ( §4). 4. We show how the †-structure of Krn can be exploited to approximate kernels by averaging ( §5). Due to an important structural feature of Krn (Th. 1) every kernel in Krn can be approximated by kernels on finite spaces. 5. We show a natural class of approximations schemes where the sequence of approximating kernels converges to the kernel to be approximated. The notion of convergence is given naturally by moving to BL σ and considering convergence in the Strong Operator Topology ( §6). 6. We apply our theory of kernel approximations to two practical applications ( §7). First, we show how Bayesian inference can be performed approximately by showing that the †-operation commutes with taking approximations. Secondly, we consider the case of ProbNetKAT, a language developed in [14,22] to probabilistically reason about networks. ProbNetKAT includes a Kleene star operator (−) * with a complex semantics which has proved hard to approximate. We show that (−) * can be approximated, and that the approximation converges.
We provide an overview of the FET-Open Project CerCo ('Certified Complexity'). Our main achievement is the development of a technique for analysing non-functional properties of programs (time, space) at the source level with little or no loss of accuracy and a small trusted code base. The core component is a C compiler, verified in Matita, that produces an instrumented copy of the source code in addition to generating object code. This instrumentation exposes, and tracks precisely, the actual (non-asymptotic) computational cost of the input program at the source level. Untrusted invariant generators and trusted theorem provers may then be used to compute and certify the parametric execution time of the code.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.