A Permutational Boltzmann Machine with Parallel Tempering for Solving Combinatorial Optimization Problems

Abstract

Boltzmann Machines are recurrent neural networks that have been used extensively in combinatorial optimization due to their simplicity and ease of parallelization. This paper introduces the Permutational Boltzmann Machine, a neural network capable of solving permutation optimization problems. We implement this network in combination with a Parallel Tempering algorithm with varying degrees of parallelism ranging from a single-thread variant to a multi-threaded system using a 64core CPU with SIMD instructions. We benchmark the performance of this new system on Quadratic Assignment Problems, using some of the most difficult known instances, and show that our parallel system performs in excess of 100× faster than any known dedicated solver, including those implemented on CPU clusters, GPUs, and FPGAs.

Experimental Evaluations of Parallel Tempering on an Ising Machine

“…Reference [13] compares the PT with simulated annealing for the traveling salesman problems and shows that the PT has a higher probability of obtaining an accurate solution with almost the same computation time. Reference [14] compares it with traditional algorithms such as the genetic algorithm on CPU and ant colony optimization on GPU for the quadratic assignment problems and shows that the optimal solution is obtained more than 100 times faster. As stated before, the METIS is a sequential algorithm running on a single core.…”

Multi-spin-flip engineering in an Ising machine

“…There is a previous study on engineering a multi-spin flip [16]. The SA performance was enhanced because simultaneous flips of multi spins can induce transitions within the subspace of FSs [16].…”

“…There is a previous study on engineering a multi-spin flip [16]. The SA performance was enhanced because simultaneous flips of multi spins can induce transitions within the subspace of FSs [16]. However, implementing the multispin flips into existing real Ising-machine hardware is quite difficult because it changes the operational principle of the Ising machine.…”

Graph clustering with Boltzmann machines

“…Our first variation of the Boltzmann machine is a one-hot encoded machine, we call this version the integer Boltzmann machine [6]. In this architecture, each one of the N vertices is represented by K binary variables representing the node's membership to each of the K clusters.…”