Controlled Experimental Design for Statistical Comparison of Integer Programming Algorithms

Abstract

Testing and comparison of integer programming algorithms is an integral part of the algorithm development process. When test problems are randomly generated, the techniques of statistical experimental design can provide a basis around which to structure computational experiments. This paper formulates the problem of constructing and analyzing controlled integer programming tests in the experimental design context and develops approaches to dealing with a number of issues that arise. Both analytic results and empirical evidence from a large experiment are employed in deriving the suggested techniques.programming: integer algorithms, statistics: design of experiments

Designing and reporting on computational experiments with heuristic methods

“…Statistical methods should be employed wherever possible to indicate the strength of relationships between di erent factors and performance measures e.g., see Amini and Barr 1990, Golden et al 1986, Lin and Rardin 1979, McGeoch 1995, Nance et al 1987 . While they cannot prove causality, these methods do indicate the reliability and validity of the results.…”

“…As a step towards changing this, it is our opinion that all doctoral students of operations research should receive training in DOE and employ i t i n any empirical research. However, see Amini and Barr 1990, Barton and Ivey, Jr. 1996, Lin and Rardin 1979, Nance et al 1987 for examples of using DOE to test optimization and heuristic algorithms.…”

An analysis of six greedy selection rules on a class of zero-one integer programming models

“…Following guidelines set forth in Lin and Rardin [5], an experimental design was developed for comparing the six algorithms described above when four model parameters of (P) are systematically varied. The parameters are the (1) number of constraints rn, (2) number of variables n, (3) constraint matrix density, and (4) constraint slackness.…”

Testing cut generators for mixed-integer linear programming

“…By carefully devising random instances with and without a particular property, one can test how the property impacts the performances of the two algorithms and test if one is significantly better than the other when the property is present in the instance. Studies of this type can be found in the literature, e.g., for the Bin Packing Problem [26], for the Generalized Assignment Problem [3], for the Computer Network Design Problem [32], for computing asymptotic estimates of algorithm complexity [30], for analysis of Integer Linear Programming algorithms [21], for Network Reoptimization Techniques [2], and for an analysis of the effect of the choice in starting points for Nonlinear Optimization Algorithms [15].…”