A Polynomial Time Algorithm for Computing Extinction Probabilities of Multitype Branching Processes

Etessami, Kousha; Stewart, Alistair; Yannakakis, Mihalis

doi:10.1137/16m105678x

Cited by 3 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[n] of target non-terminals, and a start non-terminal T i ∈ V , decide whether (i) ∃σ ∈ Ψ : P r σ Ti [ q∈K Reach(T q )] = 1, & (ii) P r * Ti [ q∈K Reach(T q )] = 1. 7 The fact that these statements are equivalent is easy to prove; see the full version. 8 Technically, as given, this OBMDP in not in simple normal form; but this can easily be rectified by using an auxiliary branching non-terminal, Q, adding the rule Q (2.)…”

Section: Propositionmentioning

confidence: 98%

Qualitative Multi-objective Reachability for Ordered Branching MDPs

Etessami

Martinov

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We study qualitative multi-objective reachability problems for Ordered Branching Markov Decision Processes (OBMDPs), or equivalently context-free MDPs, building on prior results for single-target reachability on Branching Markov Decision Processes (BMDPs). We provide two separate algorithms for "almost-sure" and "limit-sure" multi-target reachability for OBMDPs. Specifically, given an OBMDP, A, given a starting non-terminal, and given a set of target non-terminals K of size k = |K|, our first algorithm decides whether the supremum probability, of generating a tree that contains every target non-terminal in set K, is 1. Our second algorithm decides whether there is a strategy for the player to almost-surely (with probability 1) generate a tree that contains every target non-terminal in set K. The two separate algorithms are needed: we give examples showing that indeed "almost-sure" = "limitsure" for multi-target reachability in OBMDPs. Both algorithms run in time 2 O(k) • |A| O(1) , where |A| is the bit encoding length of A. Hence they run in P-time when k is fixed, and are fixed-parameter tractable with respect to k. Moreover, we show that the qualitative almost-sure (and limit-sure) multi-target reachability decision problem is in general NP-hard, when k is not fixed.

show abstract

Section: Propositionmentioning

confidence: 98%

Qualitative Multi-objective Reachability for Ordered Branching MDPs

Etessami

Martinov

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…This objective was studied in [10] (and [11]) for OBMDPs and their (concurrent) stochastic game generalizations. 8 This paper considers the multi-objective reachability problem, which is a natural extension of the previously studied (single-target) reachability problem. In the multi-objective setting we have multiple target non-terminals, and we want to optimize each of the respective probabilities of achieving multiple given objectives, each one being a boolean combination of reachability and 7 We can assume, without loss of generality, that the initial derivation consists of a single given root, because for any given collection µ ∈ V * of multiple roots, we can always add an auxiliary non-terminal T f to the set V , where Γ f = {a} and the set R(T f , a) contains a single probabilistic rule, T f 1 − → µ.…”

Section: Introductionmentioning

confidence: 99%

“…In the multi-objective setting we have multiple target non-terminals, and we want to optimize each of the respective probabilities of achieving multiple given objectives, each one being a boolean combination of reachability and 7 We can assume, without loss of generality, that the initial derivation consists of a single given root, because for any given collection µ ∈ V * of multiple roots, we can always add an auxiliary non-terminal T f to the set V , where Γ f = {a} and the set R(T f , a) contains a single probabilistic rule, T f 1 − → µ. 8 The models analysed in [10] and [11] are game generalizations of Branching Processes, but for the case of a single target computing reachability probabilities in Branching Processes is equivalent to computing reachability probabilities in Ordered Branching Processes (same holds for the MDP and game generalizations of these models).…”

Section: Introductionmentioning

confidence: 99%

Qualitative Multi-Objective Reachability for Ordered Branching MDPs

Etessami¹,

Martinov²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We study qualitative multi-objective reachability problems for Ordered Branching Markov Decision Processes (OBMDPs), or equivalently context-free MDPs, building on prior results for single-target reachability on Branching Markov Decision Processes (BMDPs).We provide two separate algorithms for "almost-sure" and "limit-sure" multi-target reachability for OBMDPs. Specifically, given an OBMDP, A, given a starting non-terminal, and given a set of target non-terminals K of size k = |K|, our first algorithm decides whether the supremum probability, of generating a tree that contains every target non-terminal in set K, is 1. Our second algorithm decides whether there is a strategy for the player to almost-surely (with probability 1) generate a tree that contains every target non-terminal in set K.The two separate algorithms are needed: we show that indeed, in this context, "almostsure" = "limit-sure" for multi-target reachability, meaning that there are OBMDPs for which the player may not have any strategy to achieve probability exactly 1 of reaching all targets in set K in the same generated tree, but may have a sequence of strategies that achieve probability arbitrarily close to 1. Both algorithms run in time 2 O(k) • |A| O(1) , where |A| is the total bit encoding length of the given OBMDP, A. Hence they run in polynomial time when k is fixed, and are fixed-parameter tractable with respect to k. Moreover, we show that even the qualitative almost-sure (and limit-sure) multi-target reachability decision problem is in general NP-hard, when the size k of the set K of target non-terminals is not fixed.

show abstract

Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations

Etessami¹,

Stewart

Yannakakis³

2020

Mathematics of OR

Self Cite

View full text Add to dashboard Cite

We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max (min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding size of the system of equations and in log(1/ǫ), where ǫ > 0 is the desired additive error bound of the solution. (The model of computation is the standard Turing machine model.) We establish this result using a generalization of Newton's method which applies to maxPPSs and minPPSs, even though the underlying functions are only piecewise-differentiable. This generalizes our recent work which provided a P-time algorithm for purely probabilistic PPSs.These equations form the Bellman optimality equations for several important classes of infinite-state Markov Decision Processes (MDPs). Thus, as a corollary, we obtain the first polynomial time algorithms for computing to within arbitrary desired precision the optimal value vector for several classes of infinite-state MDPs which arise as extensions of classic, and heavily studied, purely stochastic processes. These include both the problem of maximizing and mininizing the termination (extinction) probability of multi-type branching MDPs, stochastic context-free MDPs, and 1-exit Recursive MDPs.Furthermore, we also show that we can compute in P-time an ǫ-optimal policy for both maximizing and minimizing branching, context-free, and 1-exit-Recursive MDPs, for any given desired ǫ > 0. This is despite the fact that actually computing optimal strategies is Sqrt-Sumhard and PosSLP-hard in this setting.We also derive, as an easy consequence of these results, an FNP upper bound on the complexity of computing the value (within arbitrary desired precision) of branching simple stochastic games (BSSGs) and related infinite-state turn-based stochastic game models.

show abstract

A Polynomial Time Algorithm for Computing Extinction Probabilities of Multitype Branching Processes

Cited by 3 publications

References 34 publications

Qualitative Multi-objective Reachability for Ordered Branching MDPs

Qualitative Multi-objective Reachability for Ordered Branching MDPs

Qualitative Multi-Objective Reachability for Ordered Branching MDPs

Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations

Contact Info

Product

Resources

About