On the Generalized Lanczos Trust-Region Method

Abstract

The so-called Trust-Region Subproblem gets its name in the trust-region method in optimization and also plays a vital role in various other applications. Several numerical algorithms have been proposed in the literature for solving small-to-medium size dense problems as well as for large scale sparse problems. The Generalized Lanczos Trust-Region (GLTR) method proposed by [Gould, Lucidi, Roma and Toint, SIAM J. Optim., 9:504-525 (1999)] is a natural extension of the classical Lanczos method for the linear system to the trust-region subproblem. In this paper, we first analyze the convergence of GLTR to reveal its convergence behavior in theory and then propose new stopping criteria that can be integrated into GLTR for better numerical performance. Specifically, we develop a priori upper bounds for the convergence to both the optimal objective value as well as the optimal solution, and argue that these bounds can be efficiently estimated numerically and serve as stopping criteria for iterative methods such as GLTR. Two sets of numerical tests are presented. In the first set, we demonstrate the sharpness of the upper bounds, and for the second set, we integrate the upper bound estimate into the Fortran routine GLTR in the library GALAHAD as new stopping criteria, and test the trust-region solver TRU on the problem collection CUTEr. The numerical results show that, with the new stopping criteria in GLTR, the overall performance of TRU can be improved considerably.

Linear Constrained Rayleigh Quotient Optimization: Theory and Algorithms

“…However, this secular equation χÔλÕ 0 has a special rational form. Previous ideas in solving secular equations of similar types [2,10,21,43] can be adopted to devise a much fast method than Newton's method. Details are presented in Appendix A.…”

“…Motivated by the treatments of the trust-region subproblem [27,43], QEPmin (2.18) can be classified into two categories, namely easy case and hard case, defined as follows. Otherwise, QEPmin (2.18) is in the easy case.…”

“…Lemma 2.1 in [17] shows that y is the solution of (2.40) if and only if there exists As we have mentioned, the terms "easy" and "hard" were adopted from the treatments of the trust-region subproblem [27,43], where the term "easy" means the associated case is easy to explain, not implying the case is easy to solve, however. A more detailed connection with TRS (2.41) is as follows.…”

“…In the easy case of QEPmin (2.18), b T 0 z ¦ 0 for all minimizers Ôλ ¦ ,z ¦ Õ. By Theorem 2.4, z ¦ S 1 w ¦ for some w ¦ È R n¡m and thus g T It is known that the generalized Lanczos method does not work for TRS (2.41) in the hard case [43,Theorem 4.6]. A restarting strategy was proposed to overcome the difficulty, but it was commented that the strategy is computationally expensive for large scale problems [16,Theorem 5.8].…”

“…Our analysis is analogous to the one in [43]. We start by establishing an optimality property of v ÔkÕ , as an approximation of v ¦ , that minimizes hÔvÕ over n 0 K k ÔPAP,b 0 Õ.…”