Personalized Optimization with User's Feedback

Abstract

This paper develops an online algorithm to solve a time-varying optimization problem with an objective that comprises a known time-varying cost and an unknown function. This problem structure arises in a number of engineering systems and cyber-physical systems where the known function captures time-varying engineering costs, and the unknown function models user's satisfaction; in this context, the objective is to strike a balance between given performance metrics and user's satisfaction. Key challenges related to the problem at hand are related to (1) the time variability of the problem, and (2) the fact that learning of the user's utility function is performed concurrently with the execution of the online algorithm. This paper leverages Gaussian processes (GP) to learn the unknown cost function from noisy functional evaluation and build pertinent upper confidence bounds. Using the GP formalism, the paper then advocates timevarying optimization tools to design an online algorithm that exhibits tracking of the oracle-based optimal trajectory within an error ball, while learning the user's satisfaction function with no-regret. The algorithmic steps are inexact, to account for possible limited computational budgets or real-time implementation considerations. Numerical examples are illustrated based on a problem related to vehicle platooning.

OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression

“…Learning to optimize and regularize is a growing research topic; see [19,20,21,22,23,24,25,26] as representative works, even though they focus on slightly different problems. Additional works in the context of learning include the design of convex loss functions in, e.g., [27,28].…”

Smooth Strongly Convex Regression

“…CR is a particular class of shape-constrained regression problems, and since its first conception in the 50's, it has attracted much attention in various domains, such as statistics, economics, operations research, signal processing and control [1]- [3]. In economics, CR has been motivated by the need for approximating consumers' utility functions from empirical data [4], a task which has been recently re-considered in the context of personalized optimization with user's feedback [5].…”

Distributed Personalized Gradient Tracking with Convex Parametric Models

“…[10,11]. For these reasons, more tailored and personalized strategies are to be preferred when dealing with humans [12].…”

“…In the proposed personalized gradient tracking strategy, the dynamic gradient tracking update is interlaced with a learning mechanism to let each node learn the user's cost function U i (x), by employing noisy user's feedback in the form of a scalar quantity given by y i,t = U (x i,t ) + i,t , where x i,t is the local, tentative solution at time t and i,t is a noise term. It is worth pointing out that in this paper, we consider convex parametric models, instead of more generic non-parametric models, such as Gaussian Processes [12,[16][17][18][19][20][21], or convex regression [22,23]. The reasons for this choice stem from the fact that (i) user's functions are or can be often approximated as convex (see, e.g., [24,25] and references therein), which makes the overall optimization problem much easier to be solved; (ii) convex parametric models have better asymptotical rate bounds 2 than convex non-parametric models [22], which is fundamental when attempting at learning with scarce data; and (iii) a solid online theory already exists in the form of recursive least squares (RLS) [26][27][28][29][30][31].…”

“…Literature survey Incorporating user's satisfaction in the decision-making process is not a new concept [32], and a centralized framework with a similar structure to the one considered in this paper has been introduced in [12], even though in the context of non-parametric learning (see also references therein for a comprehensive literature survey). As mentioned, letting user's utility functions enter in optimization problems to drive the systems to a "comfortable" solution is fundamental in data-driven control and optimization problems involving both humans and machines (see [33,34]).…”