Error Stability Properties of Generalized Gradient-Type Algorithms

Abstract

Abstract. We present a unified framework for convergence analysis of generalized subgradient-type algorithms in the presence of perturbations. A principal novel feature of our analysis is that perturbations need not tend to zero in the limit. It is established that the iterates of the algorithms are attracted, in a certain sense, to an e-stationary set of the problem, where e depends on the magnitude of perturbations. Characterization of the attraction sets is given in the general (nonsmooth and nonconvex) case. The results are further strengthened for convex, weakly sharp, and strongly convex problems. Our analysis extends and unifies previously known results on convergence and stability properties of gradient and subgradient methods, including their incremental, parallel, and heavy ball modifications.

On the projected subgradient method for nonsmooth convex optimization in a Hilbert space

“…The constrained case with nonsmooth f, and ek satisfying (4) and lim~o~c~ = 0 (instead of (5)), is studied in [9]. The iterative step is given by a formula similar to (8) with u k E Of(xk), but an error term is allowed, as in [10], which we discuss below. It is assumed that Of is uniformly monotone, i.e.…”

“…The case of inexact subgradients is considered in [9,10]. The iteration in [9,10] is of the form x~+l= Pc(x ~-c~(uk+ vk)) with ukE 0f(xk).…”

“…The iteration in [9,10] is of the form x~+l= Pc(x ~-c~(uk+ vk)) with ukE 0f(xk). In [9] it is assumed that lim~_~v k = 0.…”

“…In [9] it is assumed that lim~_~v k = 0. In [10], on the other hand, the hypothesis is that I1¢11 ~< ~, i.e. an error of magnitude less that z is allowed in the computation of the subgradient.…”

“…an error of magnitude less that z is allowed in the computation of the subgradient. In [10] f is not assumed to be convex, just locally Lipschitzian, but in the convex case this algorithm is virtually identical to (7) and (8), since O~f(x) contains and is contained in the image through Of of appropriate balls around x. Ref. [10] characterizes the set of attractors of the sequence {xk}, which is shown to consist of approximate solutions of the problem.…”