Vindicating a sophisticated but non-rigorous physics approach called the cavity method, we establish a formula for the mutual information in statistical inference problems induced by random graphs and we show that the mutual information holds the key to understanding certain important phase transitions in random graph models. We work out several concrete applications of these general results. For instance, we pinpoint the exact condensation phase transition in the Potts antiferromagnet on the random graph, thereby improving prior approximate results [Contucci et al.: Communications in Mathematical Physics 2013]. Further, we prove the conjecture from [Krzakala et al.: PNAS 2007] about the condensation phase transition in the random graph coloring problem for any number q ≥ 3 of colors. Moreover, we prove the conjecture on the information-theoretic threshold in the disassortative stochastic block model [Decelle et al.: Phys. Rev. E 2011]. Additionally, our general result implies the conjectured formula for the mutual information in Low-Density Generator Matrix codes [Montanari: IEEE Transactions on Information Theory 2005].intuition on board but require extraneous assumptions (e.g., that the clause length k or the number of colors be very large). Moreover, many proofs require lengthy detours or case analyses that ought to be expendable. Hence, the obvious question is: can we vindicate the physics calculations wholesale?The main result of this paper is that for a wide class of problems within the purview of the replica symmetric cavity method the answer is 'yes'. More specifically, the cavity method reduces a combinatorial problem on a random graph to an optimization problem on the space of probability distributions on a simplex of bounded dimension. We prove that this reduction is valid under a few easy-to-check conditions. Furthermore, we verify that the stochastic optimization problem admits a combinatorial interpretation as the problem of finding an optimal set of Belief Propagation messages on a Galton-Watson tree. Thus, we effectively reduce a problem on a random graph, a mesmerizing object characterized by expansion properties, to a calculation on a random tree. This result reveals an intriguing connection between statistical inference problems and phase transitions in random graph models, specifically a phase transition that we call the information-theoretic threshold, which in many important models is identical to the so-called "condensation phase transition" predicted by physicists [55]. Moreover, the proofs provide a direct rigorous basis for the physics calculations, and we therefore believe that our techniques will find future applications. To motivate the general results about the connection between statistical inference and phase transitions, which we state in Section 2, we begin with four concrete applications that have each received considerable attention in their own right.1.1. The Potts antiferromagnet. As a first example we consider the antiferromagnetic Potts model on the Erdős-Rényi random ...
Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document. When citing, please reference the published version. Take down policy While the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has been uploaded in error or has been deemed to be commercially or otherwise sensitive.
The problem of identifying a planted assignment given a random k-SAT formula consistent with the assignment exhibits a large algorithmic gap: while the planted solution becomes unique and can be identified given a formula with O(n log n) clauses, there are distributions over clauses for which the best known efficient algorithms require n k/2 clauses. We propose and study a unified model for planted k-SAT, which captures well-known special cases. An instance is described by a planted assignment σ and a distribution on clauses with k literals. We define its distribution complexity as the largest r for which the distribution is not r-wise independent (1 ≤ r ≤ k for any distribution with a planted assignment).Our main result is an unconditional lower bound, tight up to logarithmic factors, for statistical (query) algorithms [Kea98, FGR + 12], matching known upper bounds, which, as we show, can be implemented using a statistical algorithm. Since known approaches for problems over distributions have statistical analogues (spectral, MCMC, gradient-based, convex optimization etc.), this lower bound provides a rigorous explanation of the observed algorithmic gap. The proof introduces a new general technique for the analysis of statistical query algorithms. It also points to a geometric paring phenomenon in the space of all planted assignments.We describe consequences of our lower bounds to Feige's refutation hypothesis [Fei02] and to lower bounds on general convex programs that solve planted k-SAT. Our bounds also extend to other planted k-CSP models, and, in particular, provide concrete evidence for the security of Goldreich's one-way function and the associated pseudorandom generator when used with a sufficiently hard predicate [Gol00].Boolean satisfiability and constraint satisfaction problems are central to complexity theory; they are canonical NP-complete problems and their approximate versions are also hard. Are they easier on average for natural distributions? An instance of random satisfiability is generated by fixing a distribution over clauses, then drawing i.i.d. clauses from this distribution. The average-case complexity of satisfiability problems is also motivated by its applications to models of disorder in physical systems, and to cryptography, which requires problems that are hard on average.Here we study planted satisfiability, in which an assignment is fixed in advance, and clauses are selected from a distribution defined by the planted assignment. Planted satisfiability and, more generally, random models with planted solutions appear widely in several different forms such as network clustering with planted partitions (the stochastic block model and its variants), random k-SAT with a planted assignment, and a proposed one-way function from cryptography [Gol00].It was noted in [BHL + 02] that drawing satisfied k-SAT clauses uniformly at random from all those satisfied by an assignment σ ∈ {±1} n often does not result in a difficult instance of satisfiability even if the number of observed clauses is relat...
We study the problem of determining the capacity of the binary perceptron for two variants of the problem where the corresponding constraint is symmetric. We call these variants the rectangle-binary-perceptron (RPB) and the u−functionbinary-perceptron (UBP). We show that, unlike for the usual step-function-binaryperceptron, the critical capacity in these symmetric cases is given by the annealed computation in a large region of parameter space (for all rectangular constraints and for narrow enough u−function constraints, K < K * ). We prove this fact (under two natural assumptions) using the first and second moment methods. We further use the second moment method to conjecture that solutions of the symmetric binary perceptrons are organized in a so-called frozen-1RSB structure, without using the replica method. We then use the replica method to estimate the capacity threshold for the UBP case when the u−function is wide K > K * . We conclude that full-step-replica-symmetry breaking would have to be evaluated in order to obtain the exact capacity in this case.
We develop an efficient algorithmic approach for approximate counting and sampling in the low-temperature regime of a broad class of statistical physics models on finite subsets of the lattice Z d and on the torus (Z/nZ) d . Our approach is based on combining contour representations from Pirogov-Sinai theory with Barvinok's approach to approximate counting using truncated Taylor series. Some consequences of our main results include an FPTAS for approximating the partition function of the hard-core model at sufficiently high fugacity on subsets of Z d with appropriate boundary conditions and an efficient sampling algorithm for the ferromagnetic Potts model on the discrete torus (Z/nZ) d at sufficiently low temperature. 1 n log Z T d n (λ) of the hard-core model on Z d . That is, we would like an algorithm which for any ǫ > 0 outputs a number η ∈ [f d (λ) − ǫ, f d (λ) + ǫ] and whose running time grows as slowly as possible as a function of 1/ǫ. Gamarnik and Katz [28] gave such an algorithm running in time polynomial in 1/ǫ for λ small enough that the strong spatial mixing holds; this condition implies the hard-core model is in the uniqueness regime. Adams, Briceño, Marcus, and Pavlov [1] gave a polynomial-time algorithm for approximating the free energy of the hard-core (and several other) models on Z 2 in a subset of the uniqueness regime. More interestingly, their results also apply for the hard-core and Widom-Rowlinson models on Z 2 in a subset of the non-uniqueness regime. The latter result is of a similar spirit to the results of this paper.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.