Spectral norm bounds for block Markov chain random matrices

Sanders, Jaron; Senen-Cerda, Albert

doi:10.48550/arxiv.2111.06201

Cited by 3 publications

(8 citation statements)

References 26 publications

(81 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This will be instrumental in our analysis, since indeed the Markov chains induced in BMDPs are homogenous. The results resemble the concentration results established by [40] (see Proposition 10 of their supplementary material SM1) and the subsequent improvements established by [41], but there are several key differences. One is that we keep track of the asymptotics in S and A, and another is that we consider restarts and have to deal with the absence of equilibrium assumption.…”

Section: D3 Towards Concentration Inequalities In Block Mdpsupporting

confidence: 79%

See 1 more Smart Citation

Nearly Optimal Latent State Decoding in Block MDPs

Jedra¹,

Lee²,

Proutière³

et al. 2022

Preprint

View full text Add to dashboard Cite

We investigate the problems of model estimation and reward-free learning in episodic Block MDPs. In these MDPs, the decision maker has access to rich observations or contexts generated from a small number of latent states. We are first interested in estimating the latent state decoding function (the mapping from the observations to latent states) based on data generated under a fixed behavior policy. We derive an information-theoretical lower bound on the error rate for estimating this function and present an algorithm approaching this fundamental limit. In turn, our algorithm also provides estimates of all the components of the MDP. We then study the problem of learning near-optimal policies in the rewardfree framework. Based on our efficient model estimation algorithm, we show that we can infer a policy converging (as the number of collected samples grows large) to the optimal policy at the best possible rate. Interestingly, our analysis provides necessary and sufficient conditions under which exploiting the block structure yields improvements in the sample complexity for identifying near-optimal policies. When these conditions are met, the sample complexity in the minimax reward-free setting is improved by a multiplicative factor n, where n is the number of possible contexts. * equal contributions Preprint. Under review.

show abstract

Section: D3 Towards Concentration Inequalities In Block Mdpsupporting

confidence: 79%

“…These concentration results are central in the performance analysis of our algorithms. The proof techniques used here draw inspiration from those used in SBMs, Block Markov Chains and matrix completion problems [19,30,40,41]. We adapt these techniques to our setting.…”

Section: E6 Analysis Of the Trimmed Random Matrixmentioning

confidence: 99%

Nearly Optimal Latent State Decoding in Block MDPs

Jedra¹,

Lee²,

Proutière³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We will rely on a concentration inequality from [57] which we reproduce here for the reader's convenience. Similar proofs for concentration in block Markov chains using this concentration inequality may be found in [64,65].…”

Section: Numerical Experiments On Manhattan Taxi Tripsmentioning

confidence: 64%

“…Block Markov chains are a model for dependent sequential data with an underlying community structure and have been used to develop and analyse community detection algorithms for sequential data; see [64,75]. Besides these two papers and the present paper, the only other rigorous analysis of the spectral properties of N X when X is a block Markov chain can be found in [65]. There, an asymptotic distance between the K largest singular values and the n − K smallest singular values is established.…”

Section: Introductionmentioning

confidence: 93%

“…In order to compare algorithmic performance to an information-theoretical lower bound on the detection error rate satisfied under any clustering algorithm, [64] required a sufficiently sharp upper bound to the largest singular value of N X − E[ N X ]. The singular values of N X were recently also considered in [65]. It is established there that N X has K informative singular values of size Θ( /n), and that the remaining n − K singular values are O( /n).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Singular value distribution of dense random matrices with block Markovian dependence

Sanders¹,

Werde²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

A block Markov chain is a Markov chain whose state space can be partitioned into a finite number of clusters such that the transition probabilities only depend on the clusters. Block Markov chains thus serve as a model for Markov chains with communities. This paper establishes limiting laws for the singular value distributions of the empirical transition matrix and empirical frequency matrix associated to a sample path of the block Markov chain whenever the length of the sample path is Θ(n 2 ) with n the size of the state space.The proof approach is split into two parts. First, we introduce a class of symmetric random matrices with dependence called approximately uncorrelated random matrices with variance profile. We establish their limiting eigenvalue distributions by means of the moment method. Second, we develop a coupling argument to show that this general-purpose result applies to block Markov chains.

show abstract

Detection and Evaluation of Clusters within Sequential Data

Werde¹,

Senen-Cerda²,

Gianluca³

et al. 2022

Preprint

View full text Add to dashboard Cite

Motivated by theoretical advancements in dimensionality reduction techniques we use a recent model, called Block Markov Chains, to conduct a practical study of clustering in real-world sequential data. Clustering algorithms for Block Markov Chains possess theoretical optimality guarantees and can be deployed in sparse data regimes. Despite these favorable theoretical properties, a thorough evaluation of these algorithms in realistic settings has been lacking.We address this issue and investigate the suitability of these clustering algorithms in exploratory data analysis of real-world sequential data. In particular, our sequential data is derived from human DNA, written text, animal movement data and financial markets. In order to evaluate the determined clusters, and the associated Block Markov Chain model, we further develop a set of evaluation tools. These tools include benchmarking, spectral noise analysis and statistical model selection tools. An efficient implementation of the clustering algorithm and the new evaluation tools is made available together with this paper.Practical challenges associated to real-world data are encountered and discussed. It is ultimately found that the Block Markov Chain model assumption, together with the tools developed here, can indeed produce meaningful insights in exploratory data analyses despite the complexity and sparsity of real-world data.

show abstract

Spectral norm bounds for block Markov chain random matrices

Cited by 3 publications

References 26 publications

Nearly Optimal Latent State Decoding in Block MDPs

Nearly Optimal Latent State Decoding in Block MDPs

Singular value distribution of dense random matrices with block Markovian dependence

Detection and Evaluation of Clusters within Sequential Data

Contact Info

Product

Resources

About