A Revisit of Block Power Methods for Finite State Markov Chain Applications

Abstract

According to the fundamental theory of Markov chains, under a simple connectedness condition, iteration of a Markov transition matrix P , on any random initial state probability vector, will converge to a unique stationary distribution, whose probability vector corresponds to the left eigenvector associated with the dominant eigenvalue λ 1 = 1. The corresponding convergence speed is governed by the magnitude of the second dominant eigenvalue λ 2 of P . Due to the simplicity to implement, this approach by using power iterations to approximate the dominant eigenpair, known as the power method, is often applied to estimate the stationary distributions of finite state Markov chains in many applications.In this paper, we revisit the generalized block power methods for approximating the eigenvector associated with λ 1 = 1 of a Markov chain transition matrix. Our analysis of the block power method shows that when s linearly independent probability vectors are used as the initial block, the convergence of the block power method to the stationary distribution depends on the magnitude of the (s+1)th dominant eigenvalue λ s+1 of P instead of that of λ 2 in the power method. Therefore, the block power method with block size s is particularly effective for transition matrices where |λ s+1 | is well separated from λ 1 = 1 but |λ 2 | is not. This approach is particularly useful when visiting the elements of a large transition matrix is the main computational bottleneck over matrix-vector multiplications, where the block power method can effectively reduce the total number of times to pass over the matrix. To further reduce the overall computational cost, we combine the block power method with a sliding window scheme, taking advantage of the subsequent vectors of the latest s iterations to assemble the block matrix. The sliding window scheme correlates vectors in the sliding window to quickly remove the influences from the eigenvalues whose magnitudes are smaller than |λs| to reduce the overall number of matrix-vector multiplications to reach convergence. Finally, we compare the effectiveness of these methods in a Markov chain model representing a stochastic luminal calcium release site.