The embedding problem for Markov matrices

Abstract

Characterizing whether a Markov process of discrete random variables has an homogeneous continuous-time realization is a hard problem. In practice, this problem reduces to deciding when a given Markov matrix can be written as the exponential of some rate matrix (a Markov generator). This is an old question known in the literature as the embedding problem [Elf37], which has been only solved for matrices of size 2 × 2 or 3 × 3. In this paper, we address this problem and related questions and obtain results in two different lines. First, for matrices of any size, we give a bound on the number of Markov generators in terms of the spectrum of the Markov matrix. Based on this, we establish a criterion for deciding whether a generic Markov matrix (different eigenvalues) is embeddable and propose an algorithm that lists all its Markov generators. Then, motivated and inspired by recent results on substitution models of DNA, we focus in the 4 × 4 case and completely solve the embedding problem for any Markov matrix. The solution in this case is more concise as the embeddability is given in terms of a single condition.

On equal-input and monotone Markov matrices

“…which is solved by C = 0 with c = 0 or, for any c > 0, implies (1 − c)C = 0 and hence c = 1. This leads to the two cases stated, where 1 is the only non-singular idempotent by (2).…”

“…Without loss of generality, we may assume a b. Also, observe that the function x → x(2 − x), on [0, 2], has a unique maximum at x = 1, with value 1, so x(x − 1) < 1 holds for all x ∈ [0, 1) ∪ (1,2]. Now, we can look at the three cases as follows.…”

“…where the alternative characterisations immediately follow from Fact 2.1 and Eq. (2). Let E i ∈ Mat(d, R) denote the matrix with 1 in all positions of column i and 0 everywhere else, which is an idempotent Markov matrix.…”

“…Not all M ∈ M 3, with positive determinant (and hence spectrum) can be embeddable, as there are cases with a single 0 in one position; see [5,11,1] and references therein for further examples. For d 4, the possibility of complex conjugate pairs of eigenvalues increases the complexity of the treatment, which is nevertheless possible with the recent results from [2].…”

“…A Markov matrix M is called embeddable [7,18,5,2] if it can be written as M = e Q with a rate matrix Q, which is a matrix with non-negative off-diagonal elements and vanishing row sums. A rate matrix Q is also called a Markov generator, or simply generator, because {e tQ : t 0} is a semigroup of Markov matrices with unit, and is thus a monoid; see [23] for general background on Markov chains in continuous time.…”