Scaling linear optimization problems prior to application of the simplex method

Elble, Joseph M.; Sahinidis, Nikolaos V.

doi:10.1007/s10589-011-9420-4

Cited by 25 publications

(15 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the large literature on scaling such problems, Downloaded 09/11/20 to 44.224.250.200. Redistribution subject to CCBY license no clear conclusions are available on when or how one should scale; see [8] for a recent experimental study. In any case, our use of these scalings is different from that in previous studies, where the aim of scaling has been to reduce a condition number or to speed up the convergence of an iterative method applied to the scaled matrix.…”

mentioning

confidence: 99%

Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems

Higham¹,

Pranesh²,

Zounon³

2019

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

\bfA \bfb \bfs \bft \bfr \bfa \bfc \bft. Motivated by the demand in machine learning, modern computer hardware is increasingly supporting reduced precision floating-point arithmetic, which provides advantages in speed, energy, and memory usage over single and double precision. Given the availability of such hardware, mixed precision algorithms that work in single or double precision but carry out part of a computation in half precision are now of great interest for general scientific computing tasks. Because of the limited range of half precision arithmetic, in which positive numbers lie between 6 \times 10-8 and 7 \times 10 4 , a straightforward rounding of single or double precision data into half precision can lead to overflow, underflow, or subnormal numbers being generated, all of which are undesirable. We develop an algorithm for converting a matrix from single or double precision to half precision. It first applies two-sided diagonal scaling in order to equilibrate the matrix (that is, to ensure that every row and column has \infty-norm 1), then multiplies by a scalar to bring the largest element within a factor \theta \leq 1 of the overflow level, and finally rounds to half precision. The second step ensures that full use is made of the limited range of half precision arithmetic, and \theta must be chosen to allow sufficient headroom for subsequent computations. We apply the new algorithm to GMRES-based iterative refinement (GMRES-IR), which solves a linear system Ax = b with single or double precision data by LU factorizing A in half precision and carrying out iterative refinement with the correction equations solved by GMRES preconditioned with the low precision LU factors. Previous implementations of this algorithm have used a crude conversion to half precision that our experiments show can cause slow convergence of GMRES-IR for badly scaled matrices or failure to converge at all. The new conversion algorithm computes \infty-norms of rows and columns of the matrix and its cost is negligible in the context of LU factorization. We show that it leads to faster convergence of GMRES-IR for badly scaled matrices and thereby allows a much wider class of problems to be solved. \bfK \bfe \bfy \bfw \bfo \bfr \bfd \bfs. diagonal scaling, half precision arithmetic, fp16, overflow, underflow, subnormal numbers, iterative refinement, linear system, mixed precision, GMRES, preconditioning \bfA \bfM \bfS \bfs \bfu \bfb \bfj \bfe \bfc \bft \bfc \bfl \bfa \bfs \bfs \bfi fi\bfc \bfa \bft \bfi \bfo \bfn \bfs. 65F05, 65F08, 65F35, 65F10 \bfD \bfO \bfI. 10.1137/18M1229511 1. Introduction. The landscape of scientific computing is changing, because of the growing availability and usage of low precision floating-point arithmetic. The 2008 revision of IEEE standard 754 introduced a 16-bit floating point format, known as half precision (fp16) [19]. Although defined only as a storage format, it has been widely adopted for computing, and is supported by the NVIDIA P100 and V100 GPUs and the AMD Radeon Instinct MI25 GPU. On such ha...

show abstract

mentioning

confidence: 99%

Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems

Higham¹,

Pranesh²,

Zounon³

2019

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

show abstract

“…For some problems, such as (3), the scaled constraints

D_{r} S D_{c} \overset{v}{true} = 0

may be satisfied accurately by the scaled solution

\overset{v}{true}

, but when the solution is unscaled,

v = D_{c} \overset{v}{true}

may violate S v = 0 significantly. We refer to [6] for a comprehensive study of scaling and its effects on the performance of the simplex method.…”

Section: Methodsmentioning

confidence: 99%

Robust flux balance analysis of multiscale biochemical reaction networks

et al. 2013

View full text Add to dashboard Cite

BackgroundBiological processes such as metabolism, signaling, and macromolecular synthesis can be modeled as large networks of biochemical reactions. Large and comprehensive networks, like integrated networks that represent metabolism and macromolecular synthesis, are inherently multiscale because reaction rates can vary over many orders of magnitude. They require special methods for accurate analysis because naive use of standard optimization systems can produce inaccurate or erroneously infeasible results.ResultsWe describe techniques enabling off-the-shelf optimization software to compute accurate solutions to the poorly scaled optimization problems arising from flux balance analysis of multiscale biochemical reaction networks. We implement lifting techniques for flux balance analysis within the openCOBRA toolbox and demonstrate our techniques using the first integrated reconstruction of metabolism and macromolecular synthesis for E. coli.ConclusionOur techniques enable accurate flux balance analysis of multiscale networks using off-the-shelf optimization software. Although we describe lifting techniques in the context of flux balance analysis, our methods can be used to handle a variety of optimization problems arising from analysis of multiscale network reconstructions.

show abstract

“…In other words, the expression (10) gives an upper bound on the maximum absolute change in x B relative to that of b. Similarly, we can get a relative change in x B by applying the Cauchy-Schwarz inequality to equations (5):…”

Section: The Condition Number Of a Square Matrixmentioning

confidence: 99%

“…This involves first scaling the constraints so that the maximum element is 1.0, then scaling the columns to also have a maximum element of 1.0. For details on different scaling methods for LPs, see Elble and Sahinidis [10]. After default scaling, c 2 remains unchanged, but c 1 is rescaled to 3/799988y 1 + 5/799988y 2 + z ≤ 800000/799988.…”

Section: Unscaled Infeasibilitiesmentioning

confidence: 99%

Identification, Assessment, and Correction of Ill-Conditioning and Numerical Instability in Linear and Integer Programs

Klotz¹

2014

Bridging Data and Decisions

View full text Add to dashboard Cite

The implementation of linear programming (LP) and mixed-integer programming (MIP) algorithms on finite precision computers can create numerical challenges that are not addressed in the mathematical descriptions of these algorithms given in many introductory and more advanced textbooks and courses. Rounding errors associated with finite precision can be magnified because of ill-conditioning or numerical instability, resulting in unexpected, possibly inconsistent results. This tutorial helps the optimization practitioner identify sources of ill-conditioning and numerical instability, assess the cause, and take appropriate remedial action. After discussing some finite precision computing fundamentals, it considers different measures of ill-conditioning, each one of which provides the simplest explanation of ill-conditioning on certain types of LP and MIP models. We then consider remedies for these numerical challenges: (i) optimizer parameter settings that treat the symptoms and (ii) diagnostic tactics that resolve the underlying MIP or LP issue. Keywords linear programming; integer programming; numerical stability; ill-conditioning; finite precision; numerical linear algebra; tutorials The fluttering of a butterfly's wing in Rio de Janeiro, amplified by atmospheric currents, could cause a tornado in Texas two weeks later.

show abstract

Scaling linear optimization problems prior to application of the simplex method

Cited by 25 publications

References 17 publications

Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems

Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems

Robust flux balance analysis of multiscale biochemical reaction networks

Identification, Assessment, and Correction of Ill-Conditioning and Numerical Instability in Linear and Integer Programs

Contact Info

Product

Resources

About