This is the moment for probabilistic loops

Abstract

We present a novel static analysis technique to derive higher moments for program variables for a large class of probabilistic loops with potentially uncountable state spaces. Our approach is fully automatic, meaning it does not rely on externally provided invariants or templates. We employ algebraic techniques based on linear recurrences and introduce program transformations to simplify probabilistic programs while preserving their statistical properties. We develop power reduction techniques to further simplify the polynomial arithmetic of probabilistic programs and define the theory of moment-computable probabilistic loops for which higher moments can precisely be computed. Our work has applications towards recovering probability distributions of random variables and computing tail probabilities. The empirical evaluation of our results demonstrates the applicability of our work on many challenging examples.

Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures

“…Finally, probabilistic assignments will involve only distributions depending on constant parameters. This restriction is more difficult to overcome and is shared with other tools based on moment-based techniques, such as Bartocci et al [2020] and Moosbrugger et al [2022]. This is because it is not always possible to derive how the moments change if one or more parameters of a distribution are probabilistic.…”

“…This is because it is not always possible to derive how the moments change if one or more parameters of a distribution are probabilistic. As in Moosbrugger et al [2022], this limitation can be mitigated by performing suitable reparametrizations (see Supplementary Material).…”

“…Differently from Carpenter et al [2017]; Gehr et al [2016]; Huang et al [2021], we consider programs involving only the arithmetic operations +, −, * , and ^. This assumption is common to other approaches relying on moment-based techniques such as Bartocci et al [2020] and Moosbrugger et al [2022], due to the fact that non-polynomial functions (such as the logarithm) may generate distributions that are not determined by their parameters. We remark that from expressions such as (1) and ( 2), general polynomial expressions and Boolean conditions can be obtained, respectively, by chaining state nodes and nesting conditional statements.…”

“…Hypothesis H1 is common to other works considering moment-based approximation, such as in Bartocci et al [2020] and Moosbrugger et al [2022] and is needed to guarantee that the method of moments converges to the true distribution [Billingsley 2013]. For a given program, it is possible to perform static analysis to check whether the arising distributions are determined by their moments, exploiting known results on moment determinacy (see, for example, the moment-generating function characterization in [Billingsley 2013]).…”

“…Over the years, many approaches have tackled this problem: numerical methods based on Monte Carlo Markov chain (MCMC) sampling [Chaganty et al 2013;Goodman et al 2008;Hastings 1970;Mansinghka et al 2014;Pfeffer 2001], variational inference (VI) Jordan et al 1999;Kucukelbir et al 2015], symbolic execution [Gehr et al 2016;Narayanan et al 2016;Saad et al 2021], volume computation [Filieri et al 2013;Holtzen et al 2020;Huang et al 2021], and approaches based on moment-based invariants [Barthe et al 2016;Bartocci et al 2020;Chakarov and Sankaranarayanan 2014;Katoen et al 2010;Moosbrugger et al 2022].…”