Solving Mixed Integer Programs Using Neural Networks

Abstract

Mixed Integer Programming (MIP) solvers rely on an array of sophisticated heuristics developed with decades of research to solve large-scale MIP instances encountered in practice. Machine learning offers to automatically construct better heuristics from data by exploiting shared structure among instances in the data. This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one.Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP. Neural Diving learns a deep neural network to generate multiple partial assignments for its integer variables, and the resulting smaller MIPs for un-assigned variables are solved with SCIP to construct high quality joint assignments. Neural Branching learns a deep neural network to make variable selection decisions in branch-and-bound to bound the objective value gap with a small tree. This is done by imitating a new variant of Full Strong Branching we propose that scales to large instances using GPUs. We evaluate our approach on six diverse real-world datasets, including two Google production datasets and MIPLIB, by training separate neural networks on each. Most instances in all the datasets combined have 10 3 − 10 6 variables and constraints after presolve, which is significantly larger than previous learning approaches. Comparing solvers with respect to primal-dual gap averaged over a held-out set of instances, the learning-augmented SCIP is 2× to 10× better on all datasets except one on which it is 10 5 × better, at large time limits. To the best of our knowledge, ours is the first learning approach to demonstrate such large improvements over SCIP on both large-scale real-world application datasets and MIPLIB.

A Mixed-Integer Programming Approach to Training Dense Neural Networks

“…The authors also explore non gradient-based approaches and initialized weights to accelerate convergence of gradient-based algorithms. Finally, there has been research directed towards using ANNs to solve MIPs [18]. In contrast to the previous literature, we propose methods that directly use MIP techniques for training ANNs as opposed to only being used in their evaluation.…”

Branch and Bound in Mixed Integer Linear Programming Problems: A Survey of Techniques and Trends

“…The proposed hybrid architecture improves the weak model by extracting high-level structural information at the initial point by the GNN model and preserves the original GNN model's ability to generalize to harder problems than trained on. Nair et al [79] adopted imitation learning to obtain a MILP brancher, where the GNN approach was expanded by implementing a large amount of parallel GPU computation. By mimicking an ADMM-based expert and combining the branching rule and primal heuristic, their work is advantageous over the SCIP [80] in terms of solving time on five benchmarks in real life.…”

“…A relatively large amount of literature focuses on embedding learning methods into primal heuristics, including but not limited to [79,[99][100][101][102][103][104][105]. Khalil et al [99] used binary classification to predict whether a primal heuristic would succeed at a given node.…”

“…In addition, it requires the problem to be solved in the future sufficiently similar to the past samples. Unlike the above learning primal heuristic works [99][100][101][102][103][104], Ding et al [105] and Nair et al [79] did not limit to problems of certain types and built a tripartite graph representation to extract correlations among variables, constraints and objective function without human intervention. Nair et al [79] posed the problem of predicting variable assignments as a generative modeling problem, which provides a principled way to learn on all available feasible assignments and also generates partial assignments at test time.…”

ML4CO: Is GCNN All You Need? Graph Convolutional Neural Networks Produce Strong Baselines For Combinatorial Optimization Problems, If Tuned and Trained Properly, on Appropriate Data

“…This GCNN-based approach for the first time reported results better than a solver (SCIP) that uses presolving, primal heuristics, and cuts. Later, [23] built on the GCNN method by incorporating an ADMM-based expert to scale-up the full strong branching to large instances. Other works include [26] where authors show that using the entire B&B tree can further boost the imitation learning, or [14] where the imitation learning was made faster by switching to a small MLP after the root node of the tree.…”