MIPaaL: Mixed Integer Program as a Layer

Abstract

Machine learning components commonly appear in larger decision-making pipelines; however, the model training process typically focuses only on a loss that measures average accuracy between predicted values and ground truth values. Decision-focused learning explicitly integrates the downstream decision problem when training the predictive model, in order to optimize the quality of decisions induced by the predictions. It has been successfully applied to several limited combinatorial problem classes, such as those that can be expressed as linear programs (LP), and submodular optimization. However, these previous applications have uniformly focused on problems with simple constraints. Here, we enable decision-focused learning for the broad class of problems that can be encoded as a mixed integer linear program (MIP), hence supporting arbitrary linear constraints over discrete and continuous variables. We show how to differentiate through a MIP by employing a cutting planes solution approach, an algorithm that iteratively tightens the continuous relaxation by adding constraints removing fractional solutions. We evaluate our new end-to-end approach on several real world domains and show that it outperforms the standard two phase approaches that treat prediction and optimization separately, as well as a baseline approach of simply applying decision-focused learning to the LP relaxation of the MIP. Lastly, we demonstrate generalization performance in several transfer learning tasks.

Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems

“…In case of problems with weaker relaxations, one could consider adding cutting planes prior to solving (Ferber et al 2019). Moreover, further improvements could be achieved by exploiting the fact that all previously computed solutions are valid candidates.…”

“…In case of weak MIP relaxations, one can also use a cutting plane algorithm in the root node and use the resulting tighter relaxation thereof (Ferber et al 2019). Other weaker oracles could also be used, for example setting a time-limit on an any-time solver and using the best solution found, or a node-limit on search algorithms.…”

A Surrogate Objective Framework for Prediction+Optimization with Soft Constraints

“…Following this idea, existing work developed implicit layers of argmin in neural network, including OptNet [13] for quadratic programs (QP) problems and CVXPY [14] for more general convex optimization problems. Further with linear relaxation and QP regularization, Wilder et al derived an end-to-end framework for combinatorial programs [9], which accelerates the computation by leverage the low-rank properties of decision vectors [3], and is further extended to mixed integer linear programs in MIPaaL [27]. Besides, for the relaxed LP problems, instead of differentiating KKT conditions, IntOpt [28] proposes an interior point based approach which computes gradients by differentiating homogeneous self-dual formulation.…”

Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions

“…A series of works about differentiating through CO problems , Elmachtoub and Grigas, 2020, Ferber et al, 2020 relax ILPs by adding L 1 , L 2 or log-barrier regularization terms and differentiate through the KKT conditions deriving from the application of the cutting plane or the interior-point methods. These approaches are conceptually linked to techniques for differentiating through smooth programs , Donti et al, 2017, Agrawal et al, 2019, Chen et al, 2020, Domke, 2012, Franceschi et al, 2018 that arise not only in modelling but also in hyperparameter optimization and meta-learning.…”