Computing Moments of First Passage Times to a Subset of States in Markov Chains

Abstract

This paper presents a relatively efficient and accurate method to compute the moments of first passage times to a subset of states in finite ergodic Markov chains. With the proposed method, the moment computation problem is reduced to the solution of a linear system of equations with the right-hand side governed by a novel recurrence for computing the higher-order moments. We propose using a form of the Grassmann-Taksar-Heyman (GTH) algorithm to solve these linear equations. Due to the form of the linear systems involved, the proposed method does not suffer from the drawbacks associated with GTH in a row-wise sparse implementation.

On Moments of Discrete Phase-Type Distributions

“…The proof of this result appears in the appendix of [4] and follows from various identities involving binomial coefficients and Stirling numbers [7]. In passing, we remark that the coefficient matrix, (I − P S,S ), of the linear system in (4) is nonsingular when P is ergodic (due to the irreducibility assumption in its definition).…”

“…Noticing that H with the probability mass function in (2) is also the random variable associated with time of first passage from the transient states in S to the absorbing state n S as in [4], its (i + 1)st moment is given by…”

“…Recently, an efficient and stable method to compute moments of first passage times from a subset of states classified as safe to the other states in ergodic discrete-time Markov chains (DTMCs) has been proposed in [4]. Although a recurrence for the moment vector of first passage times involving a common coefficient matrix and the previous moment vector on the right-hand side for the continuous-time case had been known for some time (see, for instance, [9]), a similar recurrence had not been given for the discrete-time case.…”

“…The difficulty in the discrete-time case lied in the fact that equations involved factorial moments rather than moments and there was no obvious way to go from factorial moments to moments. The method proposed in [4] reduced the computation of the first k moment vectors of first passage times from a subset of states to the solution of k linear systems with a common coefficient matrix and right-hand sides governed by a novel recurrence involving the binomial coefficients and the first (k − 1) moment vectors. The efficiency and stability of the proposed method rest respectively on the smaller linear systems solved with a common coefficient matrix and the particular implementation of the Grassmann-Taksar-Heyman (GTH) method [8] used in factorizing the coefficient matrix.…”

“…Since the first passage time from a given set of states to the other states in an ergodic DTMC is also the absorption time in the other states, it is possible to use the method proposed in [4] to compute moments of DPH distributions. It is our belief that moment matching techniques will benefit from the ability to compute higher moments of DPH distributions without having to resort to generating functions, transform techniques, or factorial moments.…”