An Integer Linear Programming Framework for Mining Constraints from Data

Abstract

Various structured output prediction problems (e.g., sequential tagging) involve constraints over the output space. By identifying these constraints, we can filter out infeasible solutions and build an accountable model. To this end, we present a general integer linear programming (ILP) framework for mining constraints from data. We model the inference of structured output prediction as an ILP problem. Then, given the coefficients of the objective function and the corresponding solution, we mine the underlying constraints by estimating the outer and inner polytopes of the feasible set. We verify the proposed constraint mining algorithm in various synthetic and real-world applications and demonstrate that the proposed approach successfully identifies the feasible set at scale. In particular, we show that our approach can learn to solve 9x9 Sudoku puzzles and minimal spanning tree problems from examples without providing the underlying rules. We also demonstrate results on hierarchical multi-label classification and conduct a theoretical analysis on how close the mined constraints are from the ground truth.Preprint. Under review.

Techniques for Symbol Grounding with SATNet

“…In the context of our formalism, this is analogous to having a fixed set of clauses M for a particular problem. Conversely, there exists a family of approaches that are not differentiable, but are able to learn logical constraints by example [9,20,21]. SATNet, however, sits somewhere in between these approaches, as it is both differentiable and able to learn a matrix M in order to fit some input data [7].…”

“…Digit classification is considered a solved problem, and while SAT constraint mining is more difficult, it could be argued that the differentiable aspect is no longer beneficial if the system needs supervision on its inputs to learn regardless. For instance, there exist other SAT constraint miners that are not differentiable but outperform SATNet [9]. Overall, the issue of being unable to learn to solve composite visual reasoning problems end-to-end is referred to as the Symbol Grounding Problem, and is considered one of the fundamental prerequisites for artificial intelligence to perform practical logical reasoning [8,10].…”

Modeling of Transport Problem in Linear Programming with Python (PULP)

“…In combinatorial problems in real life, the method of zero-one integer programming is used to find the optimal solution in set of feasible solutions. [9] Presented mixed integer linear programming is a framework for rapid conversion and global optimum. They presented a model and method which form the fundamentals of process integration, restriction and problem formulation.…”