A Filter and Nonmonotone Adaptive Trust Region Line Search Method for Unconstrained Optimization

Abstract

In this paper, a new nonmonotone adaptive trust region algorithm is proposed for unconstrained optimization by combining a multidimensional filter and the Goldstein-type line search technique. A modified trust region ratio is presented which results in more reasonable consistency between the accurate model and the approximate model. When a trial step is rejected, we use a multidimensional filter to increase the likelihood that the trial step is accepted. If the trial step is still not successful with the filter, a nonmonotone Goldstein-type line search is used in the direction of the rejected trial step. The approximation of the Hessian matrix is updated by the modified Quasi-Newton formula (CBFGS). Under appropriate conditions, the proposed algorithm is globally convergent and superlinearly convergent. The new algorithm shows better performance in terms of the Dolan-Moré performance profile. Numerical results demonstrate the efficiency and robustness of the proposed algorithm for solving unconstrained optimization problems. Symmetry 2020, 12, 656 2 of 22where . is the Euclidean norm, f k = f (x k ), g k = ∇ f (x k ), B k is a symmetric approximation matrix of G k = ∇ 2 f (x k ), and ∆ k is a trust region radius.Traditional trust region methods have some disadvantages, such as the fact that the subproblem needs to be solved many times to obtain an acceptable trial step within one iteration, which leads to high computational costs for the iterative process. One way to overcome this disadvantage is to use a line search strategy in the direction of the rejected trial step. Based on this situation, Nocedal and Yuan [9] proposed an algorithm in 1998, combining the trust region method and the line search method for the first time. Inspired by this, Michael et al., Li et al., and Zhang et al. proposed a trust region method with the line search strategy ([10-12], respectively).As can be seen in other works [4,7,8] monotone techniques are distinguished from nonmonotone techniques in that the value of the function needs to be reduced at each iteration; at the same time, the use of nonmonotone techniques can not only guarantee finding the global optimal solution effectively, but also improve the convergence rate of the algorithm. The watchdog technique was presented by Chamberlain et al. [13] in 1982 to overcome the Maratos effect of constrained optimization problems. Motivated by this idea, a nonmonotone line search technique was proposed by Grippo et al. [14] in 1986. The step length α k satisfies the following inequality:

A modified filter nonmonotone adaptive retrospective trust region method

“…ANTRL: ANTRL method is denoted by Ahookhosh et al in [6]; NFTR: NFTR method is denoted by [17]; WA FTR: WAFTR method is expressed by Qu et al in [15]; NAFRTR-1: Algorithm 2.1 with Δ 0 = 1; NAFRTR-2: Algorithm 2.1 with Δ 0 = 10; NAFRTR-3: Algorithm 2.1 with Δ 0 = 100; As is known to all, BFGS correction is one of the most important methods in quasi-Newton method. Several improved BFGS methods are given in [21,22], and the convergence theory has been well established.…”

“…Despite effectiveness of the Zhang’s method, calculating an estimation of the inverse of the Hessian in each iteration which results in some additional computational costs. Qu et al [ 15 ] refered to another adaptive strategy for updating the trust region radius as follows, …”