Learning from survey propagation: a neural network for MAX-E-3-SAT

Abstract

Many natural optimization problems are NP-hard, which implies that they are probably hard to solve exactly in the worst-case. However, it suffices to get reasonably good solutions for all (or even most) instances in practice. This paper presents a new algorithm for computing approximate solutions in Θ(N) for the maximum exact 3-satisfiability (MAX-E-3-SAT) problem by using supervised learning methodology. This methodology allows us to create a learning algorithm able to fix Boolean variables by using local information obtained by the Survey Propagation algorithm. By performing an accurate analysis, on random conjunctive normal form instances of the MAX-E-3-SAT with several Boolean variables, we show that this new algorithm, avoiding any decimation strategy, can build assignments better than a random one, even if the convergence of the messages is not found. Although this algorithm is not competitive with state-of-the-art maximum satisfiability solvers, it can solve substantially larger and more complicated problems than it ever saw during training.

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks

“…A deep neural network is a type of ML model, and when a deep network is fitted to data, this is referred to as deep learning [31]. Deep learning (DL) has shown very powerful empirical performance for solving very complex real-world problems in areas such as computer vision [32], natural language processing [33,34], speech recognition [35], recommendation systems [36], drug discovery [37], differential equations [38,39], and much more [40][41][42]. In simple words, DL can be seen a neural network [43], composed by many layers, that takes some data set D, input and targets, and learns the rules for forecasting new input data.…”

Large Independent Sets on Random d-Regular Graphs with Fixed Degree d

“…The MIS is important for applications in computer science, operations research, and engineering via such uses as graph coloring, assigning channels to the radio stations, register allocation in a compiler, artificial intelligence etc. [5][6][7][8].…”

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

“…If one wants to use the MC method for computing the global minimum, one can either run the algorithm at T = 0 or change slowly the temperature from an initial value to T = 0 : this is the so-called Simulated Annealing algorithm 14 . A key property that allows for a solid theory of MC is the so-called detailed balance condition that ensures the algorithm admits a limiting distribution at large times 15,16 .SGD [17][18][19] is a popular optimization algorithm used in the development of state-of-the-art machine learning 20 and deep learning models 21,22 , which have shown tremendous success in numerous fields, becoming indispensable tools for many advanced applications [23][24][25][26][27][28][29][30][31] . It is an extension of the gradient descent algorithm 32 that uses a subset of the training data to compute the gradient of the objective function at each iteration.…”

“…SGD [17][18][19] is a popular optimization algorithm used in the development of state-of-the-art machine learning 20 and deep learning models 21,22 , which have shown tremendous success in numerous fields, becoming indispensable tools for many advanced applications [23][24][25][26][27][28][29][30][31] . It is an extension of the gradient descent algorithm 32 that uses a subset of the training data to compute the gradient of the objective function at each iteration.…”