Variable Metric Inexact Line-Search-Based Methods for Nonsmooth Optimization

Abstract

We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly non differentiable, function. The key features of the proposed method are the definition of a suitable descent direction, based on the proximal operator associated to the convex part of the objective function, and an Armijo-like rule to determine the step size along this direction ensuring the sufficient decrease of the objective function. In this frame, we especially address the possibility of adopting a metric which may change at each iteration and an inexact computation of the proximal point defining the descent direction. For the more general nonconvex case, we prove that all limit points of the iterates sequence are stationary, while for convex objective functions we prove the convergence of the whole sequence to a minimizer, under the assumption that a minimizer exists. In the latter case, assuming also that the gradient of the smooth part of the objective function is Lipschitz, we also give a convergence rate estimate, showing the O( 1 k ) complexity with respect to the function values. We also discuss verifiable sufficient conditions for the inexact proximal point and we present the results of a numerical experience on a convex total variation based image restoration problem, showing that the proposed approach is competitive with another state-of-the-art method.

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

“…Throughout the entire section, {x (k) } k∈N will denote the sequence generated by SGP. The following lemma (whose proof follows from [9…”

A comparison of edge-preserving approaches for differential interference contrast microscopy

“…The parameter m in Algorithm 1 is typically a small value (m = 3, 4, 5), in order to avoid a significant computational cost in the calculation of the steplengths α (20) is addressed, at each iteration of ILA, by means of algorithm FISTA [43] which is stopped by using criterion (22) with η = 10 −6 . This value represents a good balance between convergence speed and computational time per iteration [14]. Concerning the nonlinear CG methods equipped with the strong Wolfe conditions, we use the same parameters as done in [41] and we initialize the related backtracking procedure as suggested in [24, p. 59].…”

“…Recall that the Lipschitz continuity of ∇J implies that there is α min > 0 such that the linesearch parameter α n ≥ α min (see [14,Proposition 4.2] for a proof). Then…”

“…Since the latter choice leads to the minimization of a smooth functional, we consider a limited memory gradient method, in which suitable adaptive steplength parameters are chosen to improve the convergence rate of the algorithm. As concerns the TV-based model, we address the minimization problem by means of a recently proposed linesearch-based forward-backward method able to handle the nonsmoothness of the TV functional [14].…”

The ROI CT problem: a shearlet-based regularization approach

“…However, our approach differs in the objective function, which considers shearlets instead of wavelets, and in the aim, since our goal is to compare the different regularization terms proposed to identify the model which better provides the desired features of the image to reconstruct. Moreover, for the solution of the minimization problems we exploit a very recently proposed algorithm belonging to the class of proximalgradient techniques, the variable metric inexact line-search algorithm (VMILA) [6], in place of a (split) augmented Lagrangian. VMILA is a proximal-gradient method, which enables the inexact computation of the proximal point defining the descent direction and guarantees the sufficient decrease of the objective function by means of an Armijo-like backtracking procedure.…”