Lexicographic ranking supermartingales: an efficient approach to termination of probabilistic programs

Abstract

Probabilistic programs extend classical imperative programs with real-valued random variables and random branching. The most basic liveness property for such programs is the termination property. The qualitative (aka almost-sure) termination problem given a probabilistic program asks whether the program terminates with probability 1. While ranking functions provide a sound and complete method for non-probabilistic programs, the extension of them to probabilistic programs is achieved via ranking supermartingales (RSMs). While deep theoretical results have been established about RSMs, their application to probabilistic programs with nondeterminism has been limited only to academic examples. For non-probabilistic programs, lexicographic ranking functions provide a compositional and practical approach for termination analysis of realworld programs. In this work we introduce lexicographic RSMs and show that they present a sound method for almost-sure termination of probabilistic programs with nondeterminism. We show that lexicographic RSMs provide a tool for compositional reasoning about almost sure termination, and for probabilistic programs with linear arithmetic they can be synthesized efficiently (in polynomial time). We also show that with additional restrictions even asymptotic bounds on expected termination time can be obtained through lexicographic RSMs. Finally, we present experimental results on abstractions of real-world programs to demonstrate the effectiveness of our approach.Lexicographic Ranking Supermartingales: An Efficient Approach to Termination of Probabilistic Programs 1:3Our contributions. In this work our main contributions range from theoretical foundations of lexicographic RSMs, to algorithmic approaches for them, to experimental results showing their applicability to programs. We describe our main contributions below:

Computing Expected Runtimes for Constant Probability Programs

“…Example 1 (Tortoise and Hare). The prowhile ((1, −1) • (t, h) > −1) { (t, h) = (t, h) + (1, 0) [ 6 11 ]; (t, h) = (t, h) + (1, 1) [ 1 22 ]; (t, h) = (t, h) + (1,2) [ 1 22 ]; (t, h) = (t, h) + (1,3) [ 1 22 ]; . .…”

“…For example, if A ⊆ (Z 2 ) ω consists of all infinite sequences starting with (5, 1), (6, 1), (7,6), then P Prace (5,1) (A) = 6 11 · 1 22 = 3 121 . So, if one starts with (5,1), then 3 121 is the probability that the next two states are (6, 1) and (7,6). Once a state is reached that violates the loop guard, then the probability to remain in this state is 1.…”

“…Approaches based on supermartingales (e.g., [1,5,11,13,14,18]) use mappings similar to rdw P in order to infer a realvalued term which over-approximates the expected runtime. However, in the following (non-trivial) theorem we show that our transformation is not only an over-or under-approximation, but the termination behavior and the expected runtime of P and P rdw are identical.…”

“…Many techniques to analyze (P)AST have been developed, which mostly rely on ranking supermartingales, e.g., [1,5,11,13,14,18,20,28,30]. Indeed, several of these works (e.g., [1,5,18,20]) present complete criteria for (P)AST, although (P)AST is undecidable. However, the corresponding automation of these techniques is of course incomplete.…”

A new proof rule for almost-sure termination

“…Finally, Agrawal et al [2018] have extended the ε-strict super-martingale approach to include lexicographic orderings, and present techniques for their automatic synthesis. (We explore parametrised-ε super-martingales, but not lexicographic, in McIver and Morgan [2016, Sec.…”