Graph Colouring Meets Deep Learning: Effective Graph Neural Network Models for Combinatorial Problems

Abstract

Deep learning has consistently defied state-of-the-art techniques in many fields over the last decade. However, we are just beginning to understand the capabilities of neural learning in symbolic domains. Deep learning architectures that employ parameter sharing over graphs can produce models which can be trained on complex properties of relational data. These include highly relevant N P-Complete problems, such as SAT and TSP. In this work, we showcase how Graph Neural Networks (GNN) can be engineered -with a very simple architecture -to solve the fundamental combinatorial problem of graph colouring. Our results show that the model, which achieves high accuracy upon training on random instances, is able to generalise to graph distributions different from those seen at training time. Further, it performs better than the Neurosat, Tabucol and greedy baselines for some distributions. In addition, we show how vertex embeddings can be clustered in multidimensional spaces to yield constructive solutions even though our model is only trained as a binary classifier. In summary, our results contribute to shorten the gap in our understanding of the algorithms learned by GNNs, as well as hoarding empirical evidence for their capability on hard combinatorial problems. Our results thus contribute to the standing challenge of integrating robust learning and symbolic reasoning in Deep Learning systems.

Graph Neural Networks for Maximum Constraint Satisfaction

“…Prates et al, (2019) use GNNs to learn TSP and trained on instances of the form (G, ℓ ± ε) where ℓ is the length of an optimal tour on G. They achieved good results on graphs with up to 40 nodes. Using the same idea, Lemos et al, (2019) learned to predict k-colorability of graphs scaling to larger graphs and chromatic numbers than seen during training. Yao et al, (2019) evaluated the performance of unsupervised GNNs for the MAX-CUT problem.…”

“…There is a long tradition of designing exact and heuristic algorithms for all kinds of CSPs. Our work should be seen in the context of a recently renewed interest in heuristics for NP-hard combinatorial problems based on neural networks, mostly GNNs (for example, Khalil et al, 2017 ; Selsam et al, 2019 ; Lemos et al, 2019 ; Prates et al, 2019 ).…”

“…This focus on the optimization problem allows us to train unsupervised, which is a major point of distinction between our work and recent neural approaches to Boolean satisfiability ( Selsam et al, 2019 ) and the coloring problem ( Lemos et al, 2019 ). Both approaches require supervised training and output a prediction for satisfiability or coloring number.…”

“…They achieved good results on graphs with up to 40 nodes. Using the same idea, Lemos et al, (2019) learned to predict k -colorability of graphs scaling to larger graphs and chromatic numbers than seen during training. Yao et al, (2019) evaluated the performance of unsupervised GNNs for the M ax -C ut problem.…”

Irregular Register Allocation for Translation of Test-pattern Programs

“…Although our modified PBQP solver could find a solution efficiently for our ATE benchmark programs, the PBQP problem itself is a hard problem, especially when no spill is allowed as in ATE register allocation. So, we might need an even more elaborate solver, and one promising approach would be to exploit machine learning based on deep neural networks [10,15,17,22]. Also, when it is really impossible to allocate registers for a given program, some changes in the source program, such as loop unrolling, might improve the chance of register by loosening the constraints related to the major cycles, without affecting the program correctness.…”