Online Convex Optimization with Time-Varying Constraints

Abstract

This paper considers online convex optimization with time-varying constraint functions. Specifically, we have a sequence of convex objective functions {ft(x)} ∞ t=0 and convex constraint functions {gt,i(x)} ∞ t=0 for i ∈ {1, ..., k}. The functions are gradually revealed over time. For a given ǫ > 0, the goal is to choose points xt every step t, without knowing the ft and gt,i functions on that step, to achieve a time average at most ǫ worse than the best fixed-decision that could be chosen with hindsight, subject to the time average of the constraint functions being nonpositive. It is known that this goal is generally impossible. This paper develops an online algorithm that solves the problem with O(1/ǫ 2 ) convergence time in the special case when all constraint functions are nonpositive over a common subset of R n . Similar performance is shown in an expected sense when the common subset assumption is removed but the constraint functions are assumed to vary according to a random process that is independent and identically distributed (i.i.d.) over time slots t ∈ {0, 1, 2, . . .}. Finally, in the special case when both the constraint and objective functions are i.i.d. over time slots t, the algorithm is shown to come within ǫ of optimality with respect to the best (possibly time-varying) causal policy that knows the full probability distribution.

Regret and Cumulative Constraint Violation Analysis for Distributed Online Constrained Convex Optimization

“…Fig. 2 shows that our proposed algorithms have smaller average cumulative constraint violation, which also matches the theoretical results since the standard constraint violation metric rather than the stricter metric was used in [23]- [25], [29].…”

“…In order to avoid using the upper bounds of the loss and constraint functions and their subgradients to design the algorithm parameters α t , β t , and γ t , inspired by the algorithms proposed in [24], [25], [50], we slightly modify the dual updating rule (10) as (14). As a result, the updating rule ( 8)-( 9) can be executed in a distributed manner, which is given in pseudocode as Algorithm 1.…”

“…Remark 2. Algorithm 1 can be recognized as the distributed variant of the centralized algorithm proposed in [24], [25].…”

“…Noting that there are no other distributed online algorithms to solve the considered problem due to the timevarying constraints, we compare our Algorithms 1 and 2 with the centralized algorithms with full-information feedback proposed in [23]- [25] 2 and the centralized algorithm with two-point bandit feedback proposed in [29]. Fig.…”

Decentralized Online Learning for Noncooperative Games in Dynamic Environments

“…xii,t := xi,t. Motivated by the algorithm proposed in [25], by modifying ( 18) and ( 19), a distributed online primal-dual dynamic mirror descent algorithm as in Algorithm 1 is designed to learn the variational GNE of the time-varying game Γ(V, Ωt, Jt) under partial-decision information. In order to execute Algorithm 1, at each time slot t, every player i needs to know ∇iJi,t(xi,t), gi,t(xi,t) and ∇gi,t(xi,t) rather than the full information of Ji,t and gi,t, which is similar to most online algorithms for optimization and games [16], [26]- [30].…”