An adaptive augmented Lagrangian method for large-scale constrained optimization

Abstract

In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis, Jiang, and Robinson [Math. Prog., DOI: 10.1007/s10107-014-0784-y, 2013]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our numerical experience suggests that the adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. As with traditional AL algorithms, the adaptive methods are matrix-free and thus represent a viable option for solving extreme-scale problems.

Primal-Dual Active-Set Methods for Large-Scale Optimization

“…In particular, the AL trust-region algorithm that we designed in [47,Algorithm 4] (henceforth referred to as algorithm ALTR) was used for phase one in Algorithm 2. Although ALTR represents a simplification of Algorithm 1, we claim that it provides iterates with similar properties.…”

Adaptive augmented Lagrangian methods: algorithms and practical numerical experience

“…In such situations, the poor choices of these quantities may lead to little or no improvement in the primal space and, in fact, the iterates may diverge from even a well-chosen initial iterate. The key idea for avoiding this behaviour in the algorithm proposed in [16] is to adaptively update the penalty parameter during the step computation in order to ensure that the trial step yields a sufficiently large reduction in linearized constraint violation, thus steering the optimization process steadily towards constraint satisfaction.…”

“…A new AL trust region method was recently proposed and analysed in [16]. The novel feature of that algorithm is an adaptive strategy for updating the penalty parameter inspired by techniques for performing such updates in the context of exact penalty methods [8,9,32].…”

A New Augmented Lagrangian Method for Equality Constrained Optimization with Simple Unconstrained Subproblem

“…Later Conn et al [3,4] presented a practical AL method and proved the global convergence under the LICQ condition. Since then, AL method attracted the attentions of many scholars and many variants were presented (see [5][6][7][8][9][10][11]). Up to now, there are many computer packages based on AL method, such as LANCELOT [4] and ALGENCAN [5,6].…”

“…In the past decades, AL method was fully developed. Attracted by its well performance, there are still many scholars devoted to research AL method and its applications in recent years (see [7,8,[11][12][13][14][15]). …”