Online Learning With Inexact Proximal Online Gradient Descent Algorithms

Abstract

We consider non-differentiable dynamic optimization problems such as those arising in robotics and subspace tracking. Given the computational constraints and the time-varying nature of the problem, a low-complexity algorithm is desirable, while the accuracy of the solution may only increase slowly over time. We put forth the proximal online gradient descent (OGD) algorithm for tracking the optimum of a composite objective function comprising of a differentiable loss function and a non-differentiable regularizer. An online learning framework is considered and the gradient of the loss function is allowed to be erroneous. Both, the gradient error as well as the dynamics of the function optimum or target are adversarial and the performance of the inexact proximal OGD is characterized in terms of its dynamic regret, expressed in terms of the cumulative error and path length of the target. The proposed inexact proximal OGD is generalized for application to large-scale problems where the loss function has a finite sum structure. In such cases, evaluation of the full gradient may not be viable and a variance reduced version is proposed that allows the component functions to be sub-sampled. The efficacy of the proposed algorithms is tested on the problem of formation control in robotics and on the dynamic foreground-background separation problem in video.

A Distributed Online Convex Optimization Algorithm with Improved Dynamic Regret

“…When prediction is used, P A T accumulates the prediction error. The gradient variation measure we propose in (5) is novel and different from the existing one based on the sup-norm considered in [15], [26], [28], [36]:…”

“…All these methods focus on centralized optimization problems. Distributed online gradient descent algorithms for unconstrained problems are proposed in [12], [28]- [31], while [32] analyzes the dynamic regret of the time-varying constrained case.…”

“…The dependence of the dynamic regret on the horizon T was originally removed in [33], [34] for centralized problems assuming strong convexity of the objective funtion, and then in [35] under less restrictive assumptions. For distributed problems, the dependence on T was removed in [36] provided that the sum of the local objective functions was strongly convex, though the regret bound achieved depends on the gradient path-length regularity measured by the sup-norm of the time difference of gradients, similar to [15], [26], [28]. It is clear that this term can become quite large, and it is thus of interest to see if the regret bound can be further improved.…”

Prediction-Correction Splittings for Nonsmooth Time-Varying Optimization

“…In signal processing, the estimation of timevarying signals on the basis of observations gathered online can be cast as the problem of solving a series of varying problems [4]- [6]. In robotics, path tracking and leader following problems can be cast in the framework of (1), see for example [7]- [9]. Other application domains are economics [10], smart grids [11], and non-linear optimization [12], [13].…”

On the Convergence of the Inexact Running Krasnosel’skiĭ–Mann Method

“…While (2) pertains to problems where the map T is "fixed" during the execution of the KM algorithm and it is known, this paper revisits the convergence of the KM method in case of time-varying and possibly inexact maps. This setting is motivated by recent efforts to address the design and analysis of running algorithms for time-varying optimization problems [4], [12]- [14], with particular emphasis on feedbackbased online optimization [14], [15]; additional works along these lines are in the context of online optimization (see the representative works [16]- [18] and references therein) and learning in dynamic environments [19], [20]. In a timevarying optimization setting, the underlying cost, constraints, and problem inputs may change at every step (or a few steps) of the algorithm; therefore, pertinent tasks in this case involve the derivation of results for the tracking of optimal solution trajectories.…”