Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

Abstract

Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form min. In order to solve this stochastic composition problem, we propose a class of stochastic compositional gradient descent (SCGD) algorithms that can be viewed as stochastic versions of quasi-gradient method. SCGD update the solutions based on noisy sample gradients of f v , g w and use an auxiliary variable to track the unknown quantity E w [g w (x)]. We prove that the SCGD converge almost surely to an optimal solution for convex optimization problems, as long as such a solution exists. The convergence involves the interplay of two iterations with different time scales. For nonsmooth convex problems, the SCGD achieve a convergence rate of O(k −1/4 ) in the general case and O(k −2/3 ) in the strongly convex case, after taking k samples. For smooth convex problems, the SCGD can be accelerated to converge at a rate of O(k −2/7 ) in the general case and O(k −4/5 ) in the strongly convex case. For nonconvex problems, we prove that any limit point generated by SCGD is a stationary point, for which we also provide the convergence rate analysis. Indeed, the stochastic setting where one wants to optimize compositions of expected-value functions is very common in practice. The proposed SCGD methods find wide applications in learning, estimation, dynamic programming, etc.

Multilevel Stochastic Gradient Methods for Nested Composition Optimization

“…For the case where T = 1, our results match the best known sample complexity upper-and lower-bounds. For the case where T = 2, our results improve the convergence rate from O(n 2/9 ) of the a-SCGD in [26] to O(n 2/5 ). Besides, with additional assumption that the inner level function f (T ) in (1.1) has Lipschitz continuous gradients, we obtain a convergence rate O(n 4/9 ) for two-level problems, which matches the stateof-art result achieved by ASC-PG in [28].…”

“…We have also obtained convergence and rate of convergence results for nonconvex problems. Table 1 summarizes our results and compare them with the best known ones for the single-and two-level stochastic compositional optimization problems [9,19,23,26,28]. We also provide numerical experiments with a risk-averse regression problem.…”

Stochastic Model-Based Minimization of Weakly Convex Functions

“…(1.5) and (1.6)) in optimization, see [27]. Stochastic compositional problems have also appeared in the parallel line of work [66]. There, the authors require the entire composite function to be either convex or smooth.…”

Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization

“…An established approach was to use two-level stochastic recursive algorithms with two stepsize sequences in different time scales: a slower one for updating the main decision variable x, and a faster one for tracking the value of the inner function(s). References [38,39] provide a detailed account of these techniques and existing results. In [40] these ideas were extended to multilevel problems of form (1), albeit with multiple time scales and under continuous differentiability assumptions.…”

“…Denote d k = y k − x k . By a discrete-time version of the argument that lead to (38), the decrease of the first part of the Lyapunov function can be estimated as follows:…”