Bandit Convex Optimization for Scalable and Dynamic IoT Management

Abstract

The present paper deals with online convex optimization involving both time-varying loss functions, and timevarying constraints. The loss functions are not fully accessible to the learner, and instead only the function values (a.k.a. bandit feedback) are revealed at queried points. The constraints are revealed after making decisions, and can be instantaneously violated, yet they must be satisfied in the long term. This setting fits nicely the emerging online network tasks such as fog computing in the Internet-of-Things (IoT), where online decisions must flexibly adapt to the changing user preferences (loss functions), and the temporally unpredictable availability of resources (constraints). Tailored for such human-in-the-loop systems where the loss functions are hard to model, a family of bandit online saddlepoint (BanSaP) schemes are developed, which adaptively adjust the online operations based on (possibly multiple) bandit feedback of the loss functions, and the changing environment. Performance here is assessed by: i) dynamic regret that generalizes the widely used static regret; and, ii) fit that captures the accumulated amount of constraint violations. Specifically, BanSaP is proved to simultaneously yield sub-linear dynamic regret and fit, provided that the best dynamic solutions vary slowly over time. Numerical tests in fog computation offloading tasks corroborate that our proposed BanSaP approach offers competitive performance relative to existing approaches that are based on gradient feedback.

Distributed Online Convex Optimization With Time-Varying Coupled Inequality Constraints

“…This is in agreement with (14b), (20b), and the theoretical results shown in [39], [40]. From the zoomed figures, we can see that the centralized algorithms in [39], [40] achieve smaller expected dynamic regret and constraint violation than our distributed algorithms, which is reasonable. We can also see that Algorithm 2 achieves smaller expected dynamic regret and constraint Expected dynamic regret 10 4 Algorithm 1 Algorithm 2 [39] (One-Point Sampling) [39] (Two-Point Sampling) [40] 200 Expected constraint violation 10 4 Algorithm 1 Algorithm 2 [39] (One-Point Sampling) [39] (Two-Point Sampling) [40]…”

“…Algorithm 1 can also achieve sublinear expected dynamic regret if V (x * T ) grows sublinearly. In this case, there exists a constant ν ∈ [0, 1), [24], [26]- [29], [39], [41]. Note that these papers did not consider bandit feedback for timevarying inequality constraints or did not even consider timevarying inequality constraints at all.…”

“…In the literature, there are few papers considering bandit online convex optimization with such constraints, although such constraints are common in applications. The authors of [38] studied online convex optimization with static inequality constraints and bandit feedback for constraints, while the authors of [39] studied online convex optimization with time-varying inequality constraints and bandit feedback for loss functions. The authors of [40] studied online convex optimization with time-varying inequality constraints and bandit feedback for both loss and constraint functions.…”

“…To the best of our knowledge, this is the first algorithm to solve the online convex optimization problem with time-varying inequality constraints in the one-point bandit feedback setting. An advantage of our algorithm is that the total number of rounds is not used in the algorithm, which is an improvement compared to the one-point sampling algorithms in [24], [26]- [29], [39], [41], although these paper did not consider bandit feedback for the time-varying inequality constraints or did not even consider time-varying inequality constraints at all. Sublinear expected regret and constraint violation are achieved by this algorithm if the accumulated variation of the comparator sequence also grows sublinearly.…”

On the Convergence of the Inexact Running Krasnosel’skiĭ–Mann Method

“…• Gradient errors: the gradient of the cost κ(i, s) log(1 + z(i, s)) for each exogenous traffic flow is estimated using a multi-point bandit feedback [15], [26]; the estimation error depends on the number of functional evaluations in constructing the proxy of the gradient in (24). • Solution dynamics: at each time step, the channel gain of links are generated by using a complex Gaussian random variable with mean 1 + 1 and a given variance v c for both real and imaginary parts; the transmit power for each node is a Gaussian random variable with mean 1 and a variance v p ; the exogenous traffics are random with mean [0.2, 0.3, 0.3, 0.4, 0.5, 0.2, 0.1, 0.4] and a given variance; and, the cost is perturbed by modifying a t .…”