Algorithmic Obstructions in the Random Number Partitioning Problem

Gamarnik, David; Kızıldağ, Eren C.

doi:10.48550/arxiv.2103.01369

Cited by 5 publications

(16 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By showing the presence of OGP for m−tuples of nearly equidistant points (in B n ), they established nearly tight hardness for sequential local algorithms: their results match the computational threshold modulo factors that are polylogarithmic (in k). A similar overlap pattern (for m−tuples consisting of nearly equidistant points) was also considered by Gamarnik and Kızıldag [GK21a] in the context of random number partitioning problem (NPP), where they established hardness well below the existential threshold. (It is worth noting that [GK21a] considers m−tuples where m itself also grows in n, m = ω n (1).…”

Section: Background and Related Workmentioning

confidence: 82%

“…Such SCGs are a ubiquitous feature in many algorithmic problems (with random inputs) appearing in high-dimensional statistical inference tasks and in the study of random combinatorial structures. A partial (and certainly incomplete) list of problems with an SCG includes constraint satisfaction problems [MMZ05, ART06, ACO08], optimization problems over random graphs [GS14,COE15,GS17a] and spin glass models [CGPR19, GJW20, GJ21, GJW21], number partitioning problem [GK21a], principal component analysis [BR13, LKZ15a, LKZ15b], and the "infamous" planted clique problem [Jer92, DM15, BHK + 19]; see also the introduction of [GK21a], the recent survey [Gam21]; and the references therein.…”

Section: Background and Related Workmentioning

confidence: 99%

“…This was done by Rahman and Virág [RV17] for local algorithms, and by Wein [Wei20] for low-degree polynomials; and is also our focus here (see below). A list of problems where the OGP is leveraged to rule out certain classes of algorithms includes optimization over random graphs and spin glass models [GJ21, GJW20, GJW21, HS21], number partitioning problem [GK21a], random constraint satisfaction problems [GS17b,BH21].…”

Section: Background and Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Gamarnik¹,

Kızıldağ²,

Perkins³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The binary (or Ising) perceptron is a toy model of a single-layer neural network and can be viewed as a random constraint satisfaction problem with a high degree of connectivity. The model and its symmetric variant, the symmetric binary perceptron (SBP), have been studied widely in statistical physics, mathematics, and machine learning.The SBP exhibits a dramatic statistical-to-computational gap: the densities at which known efficient algorithms find solutions are far below the threshold for the existence of solutions. Furthermore, the SBP exhibits a striking structural property: at all positive constraint densities almost all of its solutions are 'totally frozen' singletons separated by large Hamming distance [PX21,ALS21b]. This suggests that finding a solution to the SBP may be computationally intractable. At the same time, however, the SBP does admit polynomial-time search algorithms at low enough densities. A conjectural explanation for this conundrum was put forth in [BDVLZ20]: efficient algorithms succeed in the face of freezing by finding exponentially rare clusters of large size. However, it was discovered recently that such rare large clusters exist at all subcritical densities, even at those well above the limits of known efficient algorithms [ALS21a]. Thus the driver of the statisticalto-computational gap exhibited by this model remains a mystery.In this paper, we conduct a different landscape analysis to explain the algorithmic tractability of this problem. We show that at high enough densities the SBP exhibits the multi Overlap Gap Property (m−OGP), an intricate geometrical property known to be a rigorous barrier for large classes of algorithms. Our analysis shows that the m−OGP threshold (a) is well below the satisfiability threshold; and (b) matches the best known algorithmic threshold up to logarithmic factors as m → ∞. We then prove that the m−OGP rules out the class of stable algorithms for the SBP above this threshold. We conjecture that the m → ∞ limit of the m-OGP threshold marks the algorithmic threshold for the problem. Furthermore, we investigate the stability of known efficient algorithms for perceptron models and show that the Kim-Roche algorithm [KR98], devised for the asymmetric binary perceptron, is stable in the sense we consider.

show abstract

Section: Background and Related Workmentioning

confidence: 82%

Section: Background and Related Workmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Gamarnik¹,

Kızıldağ²,

Perkins³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However in these arguments, it is always possible that the structure of replicas identified is an ultrametric. Specifically, in a "star" multi-OGP [RV17, GS17,GK21b] all the replicas are pairwise equidistant. For the "ladder" OGP implementations of [Wei20,BH21], the forbidden structure is defined by applying some stopping rule to choose a finite number of solutions from a "stably evolving" sequence of algorithmic outputs.…”

Section: Necessity Of Full Branching Treesmentioning

confidence: 99%

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

Huang¹,

Sellke²

2021

Preprint

View full text Add to dashboard Cite

We study the problem of algorithmically optimizing the Hamiltonian H N of a spherical or Ising mixed p-spin glass. The maximum asymptotic value OPT of H N /N is characterized by a variational principle known as the Parisi formula, proved first by Talagrand and in more generality by Panchenko. Recently developed approximate message passing algorithms efficiently optimize H N /N up to a value ALG given by an extended Parisi formula, which minimizes over a larger space of functional order parameters. These two objectives are equal for spin glasses exhibiting a no overlap gap property. However, ALG < OPT can also occur, and no efficient algorithm producing an objective value exceeding ALG is known.We prove that for mixed even p-spin models, no algorithm satisfying an overlap concentration property can produce an objective larger than ALG with non-negligible probability. This property holds for all algorithms with suitably Lipschitz dependence on the disorder coefficients of H N . It encompasses natural formulations of gradient descent, approximate message passing, and Langevin dynamics run for bounded time and in particular includes the algorithms achieving ALG mentioned above. To prove this result, we substantially generalize the overlap gap property framework introduced by Gamarnik and Sudan to arbitrary ultrametric forbidden structures of solutions. Contents

show abstract

“…Using multi-OGPs, a line of work [69,43] culminating in the paper of Wein [71] tightly identified the algorithmic phase transition of maximum independent set on a sparse Erdős-Rényi graph for low degree polynomials. Multi-OGPs have also been used in [46,44] to rule out classes of algorithms for random k-SAT and the number partitioning problem well below the point where solutions exist, although these results are not tight against the best algorithms.…”

Section: Introductionmentioning

confidence: 99%

The Algorithmic Phase Transition of Random $k$-SAT for Low Degree Polynomials

Bresler¹,

Huang²

2021

Preprint

View full text Add to dashboard Cite

Let Φ be a uniformly random k-SAT formula with n variables and m clauses. We study the algorithmic task of finding a satisfying assignment of Φ. It is known that a satisfying assignment exists with high probability at clause density m/n < 2 k log 2 − 1 2 (log 2 + 1) + o k (1), while the best polynomial-time algorithm known, the Fix algorithm of Coja-Oghlan [25], finds a satisfying assignment at the much lower clause density (1 − o k (1))2 k log k/k. This prompts the question: is it possible to efficiently find a satisfying assignment at higher clause densities?To understand the algorithmic threshold of random k-SAT, we study low degree polynomial algorithms, which are a powerful class of algorithms including Fix, Survey Propagation guided decimation (with bounded or mildly growing number of message passing rounds), and paradigms such as message passing and local graph algorithms. We show that low degree polynomial algorithms can find a satisfying assignment at clause density (1 − o k (1))2 k log k/k, matching Fix, and not at clause density (1 + o k (1))κ * 2 k log k/k, where κ * ≈ 4.911. This shows the first sharp (up to constant factor) computational phase transition of random k-SAT for a class of algorithms. Our proof establishes and leverages a new many-way overlap gap property tailored to random k-SAT.

show abstract

Algorithmic Obstructions in the Random Number Partitioning Problem

Cited by 5 publications

References 39 publications

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

The Algorithmic Phase Transition of Random $k$-SAT for Low Degree Polynomials

Contact Info

Product

Resources

About