Application of Jacobi and Gauss–Seidel Numerical Iterative Solution Methods for the Stationary Distribution of Markov Chain

Abstract

The Physical or Mathematical behaviour of this model may be represented by describing all the different states it may occupy and by indicating how it moves among these states. In this study, the stationary distribution of Markov chains was solved using iterative methods that begin with an initial estimate of the solution vector and then modified it in a way that brings it closer and closer to the real solution with each step or iteration. These methods also involved matrix operations like multiplication with one or more vectors, which preserves the transition matrices while speeding up the process. We computed the solutions using Jacobi iterative method and Gauss-Seidel iterative method in order to shed more light on the solutions of stationary distribution in Markov chain. This was done with the aid of several already-existing laws, theorems, and formulas of Markov chain and the application of normalization principle and matrix operations such as lower, upper, and diagonal matrices. The stationary distribution vector’s 𝜋𝑖,𝑖=1,2,…,4 are obtained for the illustrative example one as 𝜋(3) = (0.078125,0.109375,0.21875,0.59375) as well as the four eigenvalues of the matrix as 𝜆1=1.0, 𝜆2=−0.7718, 𝜆3,4=−0.1141±0.5576𝑖 using Jacobi iterative technique, and for illustrative example two using Gauss-Siedel method as 𝝅(𝟑) = (0.090909, 0.181818, 0.363636, 0.363636). The research shown that Gauss Siedel method converged faster than Jacobi method.