Sequential Prediction Over Hierarchical Structures

2022

Preprint

Self Cite

In this work, we propose a computationally efficient algorithm for the problem of global optimization in univariate loss functions. For the performance evaluation, we study the cumulative regret of the algorithm instead of the simple regret between our best query and the optimal value of the objective function. Although our approach has similar regret results with the traditional lower-bounding algorithms such as the Piyavskii-Shubert method for the Lipschitz continuous or Lipschitz smooth functions, it has a major computational cost advantage. In Piyavskii-Shubert method, for certain types of functions, the query points may be hard to determine (as they are solutions to additional optimization problems). However, this issue is circumvented in our binary sampling approach, where the sampling set is predetermined irrespective of the function characteristics. For a search space of [0, 1], our approach has at most L log(3T ) and 2.25H regret for L-Lipschitz continuous and H-Lipschitz smooth functions respectively. We also analytically extend our results for a broader class of functions that covers more complex regularity conditions.

Section: A Motivationmentioning

confidence: 99%

Low Regret Binary Sampling Method for Efficient Global Optimization of Univariate Functions

Gokcesu¹,

2022

Preprint

Self Cite

“…In machine learning literature [1], [2], the area of online learning [3] is heavily investigated in various fields from game theory [4], [5], control theory [6]- [8], decision theory [9], [10] to computational learning theory [11], [12]. Because of the heavily utilized universal prediction perspective [13], it has been considerably applied in data and signal processing [14]- [19], especially in sequential prediction and estimation problems [20]- [23] such as the problem of density estimation and anomaly detection [24]- [28]. Some of its most prominent applications are in multi-agent systems [29]- [31] and specifically in reinforcement learning problems [32]- [42].…”

Section: A Preliminariesmentioning

confidence: 99%

Generalized Translation and Scale Invariant Online Algorithm for Adversarial Multi-Armed Bandits

Gokcesu¹,

2021

Preprint

Self Cite

We study the adversarial multi-armed bandit problem and create a completely online algorithmic framework that is invariant under arbitrary translations and scales of the arm losses. We study the expected performance of our algorithm against a generic competition class, which makes it applicable for a wide variety of problem scenarios. Our algorithm works from a universal prediction perspective and the performance measure used is the expected regret against arbitrary arm selection sequences, which is the difference between our losses and a competing loss sequence. The competition class can be designed to include fixed arm selections, switching bandits, contextual bandits, or any other competition of interest. The sequences in the competition class are generally determined by the specific application at hand and should be designed accordingly. Our algorithm neither uses nor needs any preliminary information about the loss sequences and is completely online. Its performance bounds are the second order bounds in terms of sum of the squared losses, where any affine transform of the losses has no effect on the normalized regret.

“…In the problems of learning, recognition, estimation or prediction [1]- [3]; decisions are often produced to minimize certain loss functions using features of the observations, which are generally noisy, random or even missing. There are numerous applications in a number of varying fields such as decision theory [4], control theory [5], game theory [6], [7], optimization [8], [9], density estimation and anomaly detection [10]- [15], scheduling [16], signal processing [17], [18], forecasting [19], [20] and bandits [21]- [23]. These decisions are acquired from specific learning models, where the goal is to distinguish certain data patterns and provide accurate estimations for practical use.…”

Section: A Calibrationmentioning

confidence: 99%

“…For example, if we join the pairs {n, n + 1} and {n + 1, n+2}, we join {n, n+1, n+2}. The new group losses and auxiliary variables are calculated in accordance with (20) and ( 21) respectively. Return to Step 3.…”

Section: B Efficient Implementationmentioning

confidence: 99%

Efficient, Anytime Algorithms for Calibration with Isotonic Regression under Strictly Convex Losses

Gokcesu¹,

2021

Preprint

Self Cite

We investigate the calibration of estimations to increase performance with an optimal monotone transform on the estimator outputs. We start by studying the traditional square error setting with its weighted variant and show that the optimal monotone transform is in the form of a unique staircase function. We further show that this staircase behavior is preserved for general strictly convex loss functions. Their optimal monotone transforms are also unique, i.e., there exist a single staircase transform that achieves the minimum loss. We propose a linear time and space algorithm that can find such optimal transforms for specific loss settings. Our algorithm has an online implementation where the optimal transform for the samples observed so far are found in linear space and amortized time when the samples arrive in an ordered fashion. We also extend our results to cases where the functions are not trivial to individually optimize and propose an anytime algorithm, which has linear space and pseudo-linearithmic time complexity.