Entropic Independence I: Modified Log-Sobolev Inequalities for Fractionally Log-Concave Distributions and High-Temperature Ising Models

Abstract

We introduce a notion called entropic independence for distributions µ defined on pure simplicial complexes, i.e., subsets of size k of a ground set of elements. Informally, we call a background measure µ entropically independent if for any (possibly randomly chosen) set S, the relative entropy of an element of S drawn uniformly at random carries at most O(1/k) fraction of the relative entropy of S, a constant multiple of its "share of entropy." Entropic independence is the natural analog of spectral independence, another recently established notion, if one replaces variance by entropy.In our main result, we show that µ is entropically independent exactly when a transformed version of the generating polynomial of µ can be upper bounded by its linear tangent, a property implied by concavity of the said transformation. We further show that this concavity is equivalent to spectral independence under arbitrary external fields, an assumption that also goes by the name of fractional log-concavity. Our result can be seen as a new tool to establish entropy contraction from the much simpler variance contraction inequalities. A key differentiating feature of our result is that we make no assumptions on marginals of µ or the degrees of the underlying graphical model when µ is based on one. We leverage our results to derive tight modified log-Sobolev inequalities for multi-step down-up walks on fractionally log-concave distributions. As our main application, we establish the tight mixing time of O(n log n) for Glauber dynamics on Ising models with interaction matrix of operator norm smaller than 1, improving upon the prior quadratic dependence on n.

Entropic Independence II: Optimal Sampling and Concentration via Restricted Modified Log-Sobolev Inequalities

“…The next proposition shows that spectral domination of µ implies a restricted form of entropic independence (see [Ana+21]), for probability distributions ν which are O(1)-bounded with respect to µ. In the special case that we can take ǫ = ∞, we do not need the O(1)-boundedness condition, and recover the fact that fractional log concavity implies entropic independence from [Ana+21].…”

“…In this work, we develop new methods for analyzing the mixing time of Markov chains and as an application, finally prove that a slight variation of Glauber dynamics, which we dub balanced Glauber dynamics, mixes in the optimal O(n log n) many steps up to the appropriate uniqueness thresholds for both the hardcore and Ising models. Our results build upon tools from recent works which analyze Markov chains through high-dimensional expansion properties [e.g., ALO21; CLV21; Che+21] and in particular, techniques for establishing entropic independence introduced by Anari, Jain, Koehler, Pham, and Vuong [Ana+21]. To surpass the limitations of previous methods, we introduce a number of new ideas for proving and using functional inequalities restricted to certain classes of well-behaved functionals.…”

“…In order to establish restricted modified log-Sobolev inequalities we use a generalization of techniques developed by earlier work of Anari, Jain, Koehler, Pham, and Vuong [Ana+21]. Roughly speaking they showed that spectral independence [ALO20], a form of variance contraction, for not just the distribution µ, but rather all external fields applied to µ, automatically entails entropic independence, a form of entropy contraction.…”

“…Previous work of [Ana+21] introduced the notion of entropic independence and new tools for proving entropic independence which we build upon in this work. In that work, one of the main results was also a O(n log n) time mixing estimate for the Ising model, established under an incomparable condition (bounded spectral norm of the interaction matrix, previously studied in [BB19; EKZ21]).…”

“…[CLV21, Fact 3.5]), where the factor of 2 comes from the different conventions we are using for ρ 0 (P). We remark that approximate tensorization of entropy is equivalent to entropy contraction of the down operator (see Definition 27), the latter notion being defined for more general distributions beyond those supported on product spaces [Ana+21].…”