Reduction of Markov Chains Using a Value-of-Information-Based Approach

Abstract

In this paper, we propose an approach to obtain reduced-order models of Markov chains. Our approach is composed of two information-theoretic processes. The first is a means of comparing pairs of stationary chains on different state spaces, which is done via the negative Kullback-Leibler divergence defined on a model joint space. Model reduction is achieved by solving a value-of-information criterion with respect to this divergence. Optimizing the criterion leads to a probabilistic partitioning of the states in the high-order Markov chain. A single free parameter that emerges through the optimization process dictates both the partition uncertainty and the number of state groups. We provide a data-driven means of choosing the 'optimal' value of this free parameter, which sidesteps needing to a priori know the number of state groups in an arbitrary chain.

Information-Theoretic Reduction of Markov Chains

“…A different approach is to define an appropriate similarity measure that directly accepts Markov chains on different state spaces. Such an approach was taken by [70]- [72]. Considering again Markov aggregation via coarse graining, the authors proposed quantities depending on the joint process (X, X) as optimization objectives.…”

“…where γ is an annealing parameter that is reduced in subsequent optimization iterations. Similarly, the authors of [72] rely on a model for the joint process (X, X) and measure similarity via a modified Kullback-Leibler divergence and the value of information, respectively. The above works require, as input, the size of the aggregated state space X .…”

An Information-theoretic Approach for Automatically Determining the Number of State Groups When Aggregating Markov Chains

“…Our corresponding technical report should be consulted for further details about these choices and illustrations of the various concepts [29]. This report also contains proofs for many of the ensuing claims.…”