Optimal Kullback-Leibler Aggregation via Spectral Theory of Markov Chains

Deng, Kun; Mehta, Prashant G.; Meyn, Sean

doi:10.1109/tac.2011.2141350

Cited by 64 publications

(126 citation statements)

References 26 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…the value function and provides algorithms which perform this aggregation [8]. Aggregation of Markov chains with information-theoretic cost functions was considered by Deng et al [3] and Vidyasagar [9], the first reference being the main inspiration of our work. Deng and Huang used the KLDR as a cost function to obtain a low-rank approximation of the original transition matrix via nuclear-norm regularization, thus preserving the cardinality of the state space [10].…”

Section: A Contributions and Related Workmentioning

confidence: 99%

“…Compared to [3], our approach differs in the definition of the lifting and its consequences. More precisely, the lifting we use incorporates the one-step transition probabilities of the original chain, while the authors of [3] define lifting based only on the stationary distribution of the original chain. Consequently, while Deng et al maximize the redundancy of the aggregated Markov chain, the lifting proposed here minimizes information loss in a welldefined sense.…”

Section: A Contributions and Related Workmentioning

confidence: 99%

“…Indeed, in stochastic modeling in computational biology [1], or in ngram word models in speech recognition [2], dealing with the state space explosion is a major challenge. Also in control theory, particularly for nearly completely decomposable Markov chains, state space reduction is an important topic [3], [4]. A direct way of reducing the state space of a Markov chain is aggregation: With the help of a partition function, groups of nodes in the original transition graph are aggregated, resulting in a graph with a smaller number of nodes.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimal Kullback–Leibler Aggregation via Information Bottleneck

Geiger

Petrov

Kubin

et al. 2015

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

Abstract-In this paper, we present a method for reducing a regular, discrete-time Markov chain (DTMC) to another DTMC with a given, typically much smaller number of states. The cost of reduction is defined as the Kullback-Leibler divergence rate between a projection of the original process through a partition function and a DTMC on the correspondingly partitioned state space. Finding the reduced model with minimal cost is computationally expensive, as it requires an exhaustive search among all state space partitions, and an exact evaluation of the reduction cost for each candidate partition. Our approach deals with the latter problem by minimizing an upper bound on the reduction cost instead of minimizing the exact cost; The proposed upper bound is easy to compute and it is tight if the original chain is lumpable with respect to the partition. Then, we express the problem in the form of information bottleneck optimization, and propose using the agglomerative information bottleneck algorithm for searching a sub-optimal partition greedily, rather than exhaustively. The theory is illustrated with examples and one application scenario in the context of modeling bio-molecular interactions.

show abstract

Section: A Contributions and Related Workmentioning

confidence: 99%

Section: A Contributions and Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Kullback–Leibler Aggregation via Information Bottleneck

Geiger

Petrov

Kubin

et al. 2015

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

show abstract

“…Although the K-L divergence does not satisfy the symmetry property and is not a metric, it provides useful interpretations for problems related to probability distributions. The usage of the K-L divergence rate as a measure of distance between two Markov chains has been widely accepted [13], [18]. Therefore, although taking d(·, ·) as the K-L divergence in (5) does not provide a metric, some notion of similarity can still be inferred.…”

Section: A Extension: When D(· ·) Is Not a P-normmentioning

confidence: 99%

A distance metric between directed weighted graphs

Salapaka

Beck

2013

52nd IEEE Conference on Decision and Control

View full text Add to dashboard Cite

Abstract-Directed weighted graphs are increasingly used to model complex systems and interactions, such as networks of interconnected physical or biological subsystems. The analysis of these graphs often requires some form of dissimilarity, or distance measure to compare graphs. In this paper, we extend connectivity-based dissimilarity measures previously used to compare unweighted undirected graphs of the same dimensions to: (1) directed weighted graphs of the same dimensions and (2) directed weighted graphs of different dimensions. To our knowledge, this is the first approach proposed for comparing two graphs containing different numbers of nodes. We derive the conditions under which this dissimilarity measure is a pseudo-metric. This derivation provides new insights on our algorithms (previously proposed) for the graph aggregation optimization problem.

show abstract

“…In [12], a singular perturbation analysis perspective on the aggregation process was provided; in [13] a simulation-based aggregation approach was presented.…”

mentioning

confidence: 99%

Aggregation of Markov chains: an analysis of deterministic annealing based methods

Xu¹,

Beck²,

Salapaka³

2014

53rd IEEE Conference on Decision and Control

View full text Add to dashboard Cite

Abstract-We develop a method for aggregating large Markov chains into smaller representative Markov chains, where Markov chains are viewed as weighted directed graphs, and similar nodes (and edges) are aggregated using a deterministic annealing approach. The notions of representativeness of the aggregated graphs and similarity between nodes in graphs are based on a newly proposed metric that quantifies connectivity in the underlying graph. Namely, we develop notions of distance between subchains in Markov chains, and provide easily verifiable conditions that determine if a given Markov chain is nearly decomposable, that is, conditions for which the deterministic annealing approach can be used to identify subchains with high probability. We show that the aggregated Markov chain preserves certain dynamics of the original chain. In particular we provide explicit bounds on the 1 norm of the error between the aggregated stationary distribution of the original Markov chain and the stationary distribution of the aggregated Markov chain, which extends on longstanding foundational results (Simon and Ando, 1961 [5]. In many cases the models comprise large graphs and networks, such as connection structures in social networks [6], metabolic networks [7], and in brain activity maps represented by Markov chains [8]. For tractable analysis and design, succinct models are necessary. Further, for large graph models, the identification of underlying coarse connectivity structure is frequently of primary interest. In this context, clustering and aggregation methods play an important role in terms of both tractability, for example in aggregation of large Markov chain models, and identification of underlying network structures.Recently, clustering based algorithms have been proposed for the purpose of determining reduced dimension graphbased models [9]. As such, an optimal aggregation of nodes in the graph is sought, where the optimality is evaluated based on a distance measure quantifying similarity in connectivity. These algorithms, which have their foundation in the deterministic annealing method proposed by Rose for the vector quantization problem [10], are directly applicable to the Markov chain reduction, or aggregation problem. In this paper, we present analytical results of graph aggregation methods specific to Markov chains; these directly generalize to weighted directed graphs.Aggregation of Markov chains, or more generally, stochastic systems, has been studied for over 50 years. One of the

show abstract

Optimal Kullback-Leibler Aggregation via Spectral Theory of Markov Chains

Cited by 64 publications

References 26 publications

Optimal Kullback–Leibler Aggregation via Information Bottleneck

Optimal Kullback–Leibler Aggregation via Information Bottleneck

A distance metric between directed weighted graphs

Aggregation of Markov chains: an analysis of deterministic annealing based methods

Contact Info

Product

Resources

About