Experimental performance of graph neural networks on random instances of max-cut

Abstract

This note explores the applicability of unsupervised machine learning techniques towards hard optimization problems on random inputs. In particular we consider Graph Neural Networks (GNNs) -a class of neural networks designed to learn functions on graphs -and we apply them to the max-cut problem on random regular graphs. We focus on the max-cut problem on random regular graphs because it is a fundamental problem that has been widely studied. In particular, even though there is no known explicit solution to compare the output of our algorithm to, we can leverage the known asymptotics of the optimal max-cut value in order to evaluate the performance of the GNNs.In order to put the performance of the GNNs in context, we compare it with the classical semidefinite relaxation approach by Goemans and Williamson (SDP), and with extremal optimization, which is a local optimization heuristic from the statistical physics literature. The numerical results we obtain indicate that, surprisingly, Graph Neural Networks attain comparable performance to the Goemans and Williamson SDP. We also observe that extremal optimization consistently outperforms the other two methods. Furthermore, the performances of the three methods present similar patterns, that is, for sparser, and for larger graphs, the size of the found cuts are closer to the asymptotic optimal max-cut value.

Graph Neural Networks for Maximum Constraint Satisfaction

“…In this section we evaluate RUN-CSP’s performance on this problem. Yao et al, (2019) proposed two unsupervised GNN architectures for M ax -C ut . One was trained through policy gradient descent on a non-differentiable loss function while the other used a differentiable relaxation of this loss.…”

“…We use their results as well as their baseline results for Extremal Optimization (EO) ( Boettcher and Percus, 2001 ) and a classical approach based on semi-definite programming (SDP) ( Goemans and Williamson, 1995 ) as baselines for RUN-CSP. To evaluate the sizes of graph cuts, Yao et al, (2019) introduced a relative performance measure called P-value given by where z is the predicted cut size for a d -regular graph with n nodes. Based on results of Dembo et al, (2017) , they showed that the expected P -value of d -regular graphs approaches as .…”

“…We use their results as well as their baseline results for Extremal Optimization (EO) (Boettcher and Percus, 2001) and a classical approach based on semi-definite programming (SDP) (Goemans and Williamson, 1995) as baselines for RUN-CSP. To evaluate the sizes of graph cuts, Yao et al, (2019) introduced a relative performance measure called P-value given by P(z) z/n−…”

“…Using the same idea, Lemos et al, (2019) learned to predict k -colorability of graphs scaling to larger graphs and chromatic numbers than seen during training. Yao et al, (2019) evaluated the performance of unsupervised GNNs for the M ax -C ut problem. They adapted a GNN architecture by Chen et al, (2019) to M ax -C ut and trained two versions of their network, one through policy gradient descent and the other via a differentiable relaxation of the loss function which both achieved similar results.…”

Scalars are universal: Equivariant machine learning, structured like classical physics

“…Implementating a universally approximating graph neural network using the formulation from the previous section would be prohibitively expensive. There is work characterizing the expressive power of message passing neural networks, mainly in terms of the graph isomorphism problem [80,57,14,50,73,12], and there is research on the design of graph networks that are expressive enough to perform specific tasks, like solving specific combinatorial optimization problems [6,11,60,38,81,37,8].…”