Constrained minimization methods

A new method is introduced for large scale convex constrained optimization. The general model algorithm involves, at each iteration, the approximate minimization of a convex quadratic on the feasible set of the original problem and global convergence is obtained by means of nonmonotone line searches. A specific algorithm, the Inexact Spectral Projected Gradient method (ISPG), is implemented using inexact projections computed by Dykstra's alternating projection method and generates interior iterates. The ISPG method is a generalization of the Spectral Projected Gradient method (SPG), but can be used when projections are difficult to compute. Numerical results for constrained least-squares rectangular matrix problems are presented.

show abstract

“…Compute y ℓ+1 using Dykstra's algorithm (22), c ℓ+1 by (25), a ℓ+1 by (34), and x ℓ+1 k by (30) for x = x k .…”

Section: Algorithm 31: Compute Approximate Projectionmentioning

confidence: 99%

“…The SPG method is related to the practical version of Bertsekas [3] of the classical gradient projected method of Goldstein, Levitin and Polyak [21,25]. However, some critical differences make this method much more efficient than its gradient-projection predecessors.…”

Section: Introductionmentioning

confidence: 99%

Inexact spectral projected gradient methods on convex sets

Birgin¹

2003

IMA Journal of Numerical Analysis

163

178

View full text Add to dashboard Cite

show abstract

“…Hence the NPPD algorithm without the line search is exactly the gradient projection algorithm [Gol64], [LeP66]. If we let W(x,y) = Vf(x) + x/c instead, then yr is given by yr = argminxx{ f(x) + Ilx -xrll 2 /2c }, so that the NPPD algorithm without the line search is exactly the proximal minimization algorithm [Mar70], [Roc76a].…”

Section: Applicationsmentioning

confidence: 99%

“…A special case of this dual method is the dual gradient method [Pan86,§6]. This algorithm is also closely related to a splitting algorithm of Gabay [Gab83] and a gradient projection algorithm of Goldstein and Levitin-Poljak [Gol64], [LeP66] -the main difference being that an additional line search is used at every iteration.…”

Section: Introductionmentioning

confidence: 99%

Decomposition algorithm for convex differentiable minimization

Tseng

1991

J Optim Theory Appl

View full text Add to dashboard Cite

In this paper we propose a decomposition algorithm for convex differentiable minimization. This algorithm at each iteration solves a variational inequality problem obtained by approximating the gradient of the cost function by a strongly monotone function. A line search is then performed in the direction of the solution to this variational inequality (with respect to the original cost). If the constraint set is a Cartesian product of m sets, the variational inequality decomposes into m coupled variational inequalities which can be solved in either a Jacobi manner or a Gauss-Seidel manner. This algorithm also applies to the minimization of strongly convex (possibly nondifferentiable) costs subject to linear constraints. As special cases, we obtain the GP-SOR algorithm of Mangasarian and De Leone, a diagonalization algorithm of Feijoo and Meyer, the coordinate descent method, and the dual gradient method. This algorithm is also closely related to a splitting algorithm of Gabay and a gradient projection algorithm of Goldstein and Levitin-Poljak, and has interesting applications to separable convex programming and to solving traffic assignment problems.

show abstract

“…Other methods require additional computation, such as a line search, to generate x i+i [ 7,19,20,30]. Furthern~re, the methods of [13,16,21 -~~~~~~ at the current point . Methods that use quasi-Newton updates [5,6,22J are not one-point methods, since the Hessian approximation depends on previous estimates of a K.K.T.…”

Section: Introductionmentioning

confidence: 99%

Globally Convergent Algorithms for Convex Programming

Rosenberg¹

1981

Mathematics of OR

View full text Add to dashboard Cite

We consider solving a (minimization) convex program by sequentially solving a (minimization) convex approximating subproblem and then executing a line search on an exact penalty function. Each subproblem is constructed from the current estimate of a solution of the given problem, possibly together with other information. Under mild conditions, solving the current subproblem generates a descent direction for the exact penalty function. Minimizing the exact penalty function along the current descent direction provides a new estimate of a solution, and a new subproblem is formed. For any arbitrary starting estimate, this scheme generates a sequence of estimates that converges to a solution of the given problem. Moreover, the functions defining the given problem and each subproblem need not be differentiable.

show abstract

Constrained minimization methods

Cited by 669 publications

References 14 publications

Inexact spectral projected gradient methods on convex sets

Inexact spectral projected gradient methods on convex sets

Decomposition algorithm for convex differentiable minimization

Globally Convergent Algorithms for Convex Programming

Contact Info

Product

Resources

About