Randomized Gradient-Free Distributed Online Optimization with Time-Varying Cost Functions

Abstract

This paper presents a randomized gradient-free distributed online optimization algorithm, with a group of agents whose local objective functions are time-varying. It is worth noting that the value of the local objective function is only revealed to the corresponding agent after the decision is made at each time-step. Thus, each agent updates the decision variable using the local objective function value of its last decision and the information collected from its immediate in-neighbors. A randomized gradient-free oracle is built locally in replacement of the true gradient information in guiding the updates of the decision variable. The notion of dynamic regret is brought forward to measure the difference between the total cost incurred by the agent's state estimation and the offline centralized optimal solution where the objective functions are available a priori. Under the assumptions of strongly connected communication graph and bounded subgradients of the local objective functions, we characterize the dynamic regret associated with each agent as a function of the time duration T and the deviation of the minimizer sequence. Averaging the dynamic regret over the time duration, we establish the asymptotic convergence to a small neighborhood of zero with a rate of O(ln T / √ T ). The effectiveness of this algorithm is illustrated through numerical simulations.

Distributed Online Optimization in Time-Varying Unbalanced Networks without Explicit Subgradients

“…In fact, ĝi,t (x) is an unbiased gradient estimator of fi,t (x). Notably, Gaussian random variables are used to construct zeroth-order oracles in [28], [35], [41], which cannot be applied in our setting. The reason is that Gaussian random variables do not have finite support such that the perturbation x + µζ i,t may lie outside of Ω.…”

“…Extending distributed algorithms from weight balanced networks to general directed networks is non-trivial [19]- [26]. The authors of [27], [28] proposed distributed online optimization algorithms inspired by the push-sum based algorithm [19] and the surplus-based method [24], respectively. However, the former is incapable of tackling constrained optimization problems by combining with projected-based methods directly, and the latter involves a global parameter depending on weight matrices which should be known a priori.…”

“…The intuition of these distributed algorithms is constructing gradient estimators from function values and then substituting them for the true gradient under the assumption of doubly stochastic weight matrices or fixed networks. Though some efforts have been made to extend zeroth-order distributed algorithms to the online setting [11], [15], [17], [18], [39] Our work [12], [28], they can only be applicable to fixed networks.…”

“…(3) In comparison with the distributed online optimization algorithms [10], [16], [18], [39] relying highly on double stochastic weight matrices, our algorithm can be applicable to unbalanced networks with column-stochastic weight matrices, and allow nodes to utilize zeroth-order information in lieu of subgradients as well. In addition, compared with the fixed networks and static regret considered in [28], our algorithm can be applicable to time-varying networks with dynamic regret guarantees. The rest of this paper is organized as follows.…”

Dynamic and Distributed Online Convex Optimization for Demand Response of Commercial Buildings

“…There is a limited but important body of work on dynamic and distributed OCO. These include work on mirror updates [18], [19], adaptive search directions [20], gradientfree methods [21] and time-varying constraints [22]. This paper extends this body of work by using a distributed weighted dual averaging update [1], [23] in the dynamic setting.…”