The Deviation Matrix of a Continuous-Time Markov Chain

Abstract

Abstract. The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix P (.) and ergodic matrix Π is the matrixWe give conditions for D to exist and discuss properties and a representation of D. The deviation matrix of a birth-death process is investigated in detail. We also describe a new application of deviation matrices by showing that a measure for the convergence to stationarity of a stochastically increasing Markov chain can be expressed in terms of the elements of the deviation matrix of the chain.

The roles of coupling and the deviation matrix in determining the value of capacity in M/M/1/C queues

“…For an irreducible, positive-recurrent, continuous-time Markov chain on the state space S with generator Q, this matrix was studied by Coolen-Schrijner and van Doorn in [6]. It is the matrix whose (i, j)th element is…”

“…see [6]. Conversely, Equation (3.3) allows us to express the mean first passage times in terms of the entries of the deviation matrix:…”

“…One involves the derivation of Markov renewal equations, conditioning on the first instance at which the state of the queue changes, a second involves expressing the value of capacity in terms of the entries of a transient analogue of the deviation matrix, discussed by Coolen-Schrijner and van Doorn in [6], and a third involves an elegant coupling argument.…”

“…First, in Section 2, we shall employ a Markov renewal analysis similar to that used in [5] for the M/M/C/C queue. Then, in Section 3, we shall relate this to a transient analogue of the deviation matrix defined in Coolen-Schrijner and van Doorn [6], at the same time deriving a number of interesting properties of this matrix. In Section 4, we shall adopt a completely different approach, coupling the evolution of M/M/1/C queues with different initial numbers of customers.…”

Series Expansions for Finite-State Markov Chains

“…where P m¼0 1 (P 2 P P ) m is often referred to as the group inverse; see, for instance, [2,4]. A general definition that is valid for any possibly periodic Markov chain can be found in, [14], for example.…”