In the Sherrington-Kirkpatrick (SK) and related mixed p-spin models, there is interest in understanding replica symmetry breaking at low temperatures. For this reason, the so-called AT line proposed by de Almeida and Thouless as a sufficient (and conjecturally necessary) condition for symmetry breaking, has been a frequent object of study in spin glass theory. In this paper, we consider the analogous condition for the multi-species SK model, which concerns the eigenvectors of a Hessian matrix. The analysis is tractable in the two-species case with positive definite variance structure, for which we derive an explicit AT temperature threshold. To our knowledge, this is the first nonasymptotic symmetry breaking condition produced for a multi-species spin glass. As possible evidence that the condition is sharp, we draw further parallel with the classical SK model and show coincidence with a separate temperature inequality guaranteeing uniqueness of the replica symmetric critical point.2010 Mathematics Subject Classification. 60K35, 82B26, 82B44.

Modern machine learning models are often so complex that they achieve vanishing classification error on the training set. Max-margin linear classifiers are among the simplest classification methods that have zero training error (with linearly separable data). Despite this simplicity, their high-dimensional behavior is not yet completely understood. We assume to be given i.i.d. data (yi, xi), i ≤ n with xi ∼ N(0, Σ) a p-dimensional Gaussian feature vector, and yi ∈ {+1, −1} a label whose distribution depends on a linear combination of the covariates θ * , xi . We consider the proportional asymptotics n, p → ∞ with p/n → ψ, and derive exact expressions for the limiting prediction error. Our asymptotic results match simulations already when n, p are of the order of a few hundreds.We explore several choices for the the pair (θ * , Σ), and show that the resulting generalization curve (test error error as a function of the overparametrization ratio ψ = p/n) is qualitatively different, depending on this choice. In particular we consider a specific structure of (θ * , Σ) that captures the behavior of nonlinear random feature models or, equivalently, two-layers neural networks with random first layer weights. In this case, we observe that the test error is monotone decreasing in the number of parameters. This finding agrees with the recently developed 'double descent' phenomenology for overparametrized models.

In a broad class of sparse random constraint satisfaction problems (csp), deep heuristics from statistical physics predict that there is a condensation phase transition before the satisfiability threshold, governed by one-step replica symmetry breaking (1rsb). In fact, in random regular knae-sat, which is one of such random csps, it was verified [45] that its free energy is well-defined and the explicit value follows the 1rsb prediction. However, for any model of sparse random csp, it has been unknown whether the solution space indeed condensates on O(1) clusters according to the 1rsb prediction. In this paper, we give an affirmative answer to this question for the random regular k-nae-sat model, by demonstrating that most of the solutions lie inside a bounded number of solution clusters whose sizes are comparable to the scale of the free energy. Furthermore, we establish that the overlap between two independently drawn solutions concentrates precisely at two values, thus proving that the nature of this condensation is of 1rsb. Contents 1. Introduction 1 2. The combinatorial model 10 3. The first moment 29 4. The second moment 51 5. The resampling method 60 6. From constant to high probability 81 7. Concentration of the overlap at two values 107 Acknowledgements 113 References 113 Appendix A. A priori estimates 116 Appendix B. Compatibility properties 139 Appendix C. Continuity of tree optimization 144 Appendix D. The second moment in the correlated regime 159 Appendix E. The effect of short cycles 162

No abstract

No abstract