A Study of Learning Search Approximation in Mixed Integer Branch and Bound: Node Selection in SCIP

Abstract

In line with the growing trend of using machine learning to help solve combinatorial optimisation problems, one promising idea is to improve node selection within a mixed integer programming (MIP) branch-and-bound tree by using a learned policy. Previous work using imitation learning indicates the feasibility of acquiring a node selection policy, by learning an adaptive node searching order. In contrast, our imitation learning policy is focused solely on learning which of a node’s children to select. We present an offline method to learn such a policy in two settings: one that comprises a heuristic by committing to pruning of nodes; one that is exact and backtracks from a leaf to guarantee finding the optimal integer solution. The former setting corresponds to a child selector during plunging, while the latter is akin to a diving heuristic. We apply the policy within the popular open-source solver SCIP, in both heuristic and exact settings. Empirical results on five MIP datasets indicate that our node selection policy leads to solutions significantly more quickly than the state-of-the-art precedent in the literature. While we do not beat the highly-optimised SCIP state-of-practice baseline node selector in terms of solving time on exact solutions, our heuristic policies have a consistently better optimality gap than all baselines, if the accuracy of the predictive model is sufficient. Further, the results also indicate that, when a time limit is applied, our heuristic method finds better solutions than all baselines in the majority of problems tested. We explain the results by showing that the learned policies have imitated the SCIP baseline, but without the latter’s early plunge abort. Our recommendation is that, despite the clear improvements over the literature, this kind of MIP child selector is better seen in a broader approach to using learning in MIP branch-and-bound tree decisions.

Learning to Compare Nodes in Branch and Bound with Graph Neural Networks

“…Finally, and more recently, Yilmaz and Yorke-Smith [35] proposed to learn a limited form of feedforward neural network node comparison operator that decides whether the branch-and-bound algorithm should expand the left child, right child or both children of a node. This operator can then be combined with a backtracking algorithm to provide a full node selection policy: in effect, this can be interpreted by combining the neural network node comparator of Song et al with a node selection rule that only calls it on children of the current node, and reverts to depth-first search otherwise.…”

“…This list is then used to select the next node to subdivide, either by simply choosing the highest-ranked node or through some more complex paradigm. Interestingly, a few works have proposed to use machine learning methods to derive node comparison functions [22,33,35]. This is particularly promising since the problem is naturally amenable to statistical learning methods.…”

“…This node representation allows complete information regarding the nodes to be provided to the model, reducing the amount of manual feature engineering. In line with previous work on this node comparison problem [22,33,35], we train the network using imitation learning to approximate a diving oracle that plunges towards the optimal solution. We compare our GNN approach against the support vector machine approach of He et al [22], the feedforward neural network approach of Song et al [33] and Yilmaz and Yorke-Smith [35], and the default node selection rule of the open-source solver SCIP [16].…”

“…In line with previous work on this node comparison problem [22,33,35], we train the network using imitation learning to approximate a diving oracle that plunges towards the optimal solution. We compare our GNN approach against the support vector machine approach of He et al [22], the feedforward neural network approach of Song et al [33] and Yilmaz and Yorke-Smith [35], and the default node selection rule of the open-source solver SCIP [16]. In addition, we compare against the node comparator of this same branching rule but with a highest-rank node selection rule.…”

Learning Variable Activity Initialisation for Lazy Clause Generation Solvers

“…Neuro-symbolic approaches to combinatorial optimisation problems include improving optimisation solver performance or robustness by incorporating machine learning (ML). This trend shows successful promise in integer programming [2,10,14,26], propositional satisfiability (SAT) [22,25] as well as constraint programming (CP) [1,9,24]. Hybrid CP-SAT solvers are the state of the art for CP according to recent MiniZinc Challenge competitions [19].…”