A Random Walk Approach to First-Order Stochastic Convex Optimization

Vakili, Sattar; Zhao, Qing

doi:10.1109/isit.2019.8849396

Cited by 10 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the inequality holds by the assumption that ǫ i are R−sub-Gaussian. As a result of Chernoff-Hoeffding inequality for sub-Gaussian distributions (Vakili and Zhao, 2019),…”

Section: Discussion and Future Workmentioning

confidence: 99%

Scalable Thompson Sampling using Sparse Gaussian Process Models

Vakili¹,

Moss²,

Артемьев³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Thompson Sampling (TS) with Gaussian Process (GP) models is a powerful tool for optimizing non-convex objective functions. Despite favorable theoretical properties, the computational complexity of the standard algorithms quickly becomes prohibitive as the number of observation points grows. Scalable TS methods can be implemented using sparse GP models, but at the price of an approximation error that invalidates the existing regret bounds. Here, we prove regret bounds for TS based on approximate GP posteriors, whose application to sparse GPs shows a drastic improvement in computational complexity with no loss in terms of the order of regret performance. In addition, an immediate implication of our results is an improved regret bound for the exact GP-TS. Specifically, we show an Õ( √ γ T T ) bound on regret that is an O( √ γ T ) improvement over the existing results where T is the time horizon and γ T is an upper bound on the information gain. This improvement is important to ensure sublinear regret bounds.

show abstract

“…where the inequality holds by the assumption that ǫ i are R−sub-Gaussian. As a result of Chernoff-Hoeffding inequality for sub-Gaussian distributions (Vakili and Zhao, 2019),…”

Section: Discussion and Future Workmentioning

confidence: 99%

Scalable Thompson Sampling using Sparse Gaussian Process Models

Vakili¹,

Moss²,

Артемьев³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Roughly speaking, this upper bound holds since the fixed size internal tests and the leaf tests have a greater probability of moving towards the anomaly than away from it. Summing the upper bound on the last passage times yields the first term in (11).…”

Section: B Performance Analysismentioning

confidence: 99%

“…The recent studies [10]- [12] considered hierarchical search under unknown observation models. The key difference is that the search strategies in [10], [11] are based on a sample mean statistic, which fails to detect a general anomalous distribution with a mean close to the mean of the normal distribution. The work in [12] does not assume a structure on the abnormal distribution, and uses the Kolmogorov-Smirnov statistic, which fails to utilize the parametric information considered in our setting.…”

Section: Introductionmentioning

confidence: 99%

Composite Anomaly Detection via Hierarchical Dynamic Search

Wolff¹,

Gafni²,

Revach³

et al. 2022

Preprint

View full text Add to dashboard Cite

Anomaly detection among a large number of processes arises in many applications ranging from dynamic spectrum access to cybersecurity. In such problems one can often obtain noisy observations aggregated from a chosen subset of processes that conforms to a tree structure. The distribution of these observations, based on which the presence of anomalies is detected, may be only partially known. This gives rise to the need for a search strategy designed to account for both the sample complexity and the detection accuracy, as well as cope with statistical models that are known only up to some missing parameters. In this work we propose a sequential search strategy using two variations of the Generalized Log Likelihood Ratio statistic. Our proposed Hierarchical Dynamic Search (HDS) strategy is shown to be order-optimal with respect to the size of the search space and asymptotically optimal with respect to the detection accuracy. An explicit upper bound on the error probability of HDS is established for the finite sample regime. Extensive experiments are conducted, demonstrating the performance gains of HDS over existing methods. B. Wolff and G. Revach are with the Institute for Signal and Information Processing (ISI), D-

show abstract

“…Referred to as Random Walk on a Tree (RWT), this policy was proposed by two of the authors of this paper in a prior work [19] that analyzed its regret performance for convex functions under sub-Gaussian noise distributions. In this paper, we demonstrate the adaptivity of RWT to different function characteristics and robustness to heavy-tailed noise with infinite variance.…”

Section: Rwt: An Adaptive and Robust Approachmentioning

confidence: 99%

“…where the second step follows from k 2/b log log k β being increasing in k. For the RHS, we can use the bound obtained in (19). Let λ(m, θ) denote the value of bound in (19)…”

Section: Appendix Cmentioning

confidence: 99%

Stochastic Gradient Descent on a Tree: an Adaptive and Robust Approach to Stochastic Convex Optimization

Vakili¹,

Salgia

Zhao

2019

2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

Self Cite

View full text Add to dashboard Cite

Online minimization of an unknown convex function over the interval [0, 1] is considered under first-order stochastic bandit feedback, which returns a random realization of the gradient of the function at each query point. Without knowing the distribution of the random gradients, a learning algorithm sequentially chooses query points with the objective of minimizing regret defined as the expected cumulative loss of the function values at the query points in excess to the minimum value of the function. An approach based on devising a biased random walk on an infinite-depth binary tree constructed through successive partitioning of the domain of the function is developed. Each move of the random walk is guided by a sequential test based on confidence bounds on the empirical mean constructed using the law of the iterated logarithm. With no tuning parameters, this learning algorithm is robust to heavy-tailed noise with infinite variance and adaptive to unknown function characteristics (specifically, convex, strongly convex, and nonsmooth). It achieves the corresponding optimal regret orders (up to a √ log T or a log log T factor) in each class of functions and offers better or matching regret orders than the classical stochastic gradient descent approach which requires the knowledge of the function characteristics for tuning the sequence of step-sizes.T t=1 (F (x t , ξ t ) − f (x * )) .

show abstract

A Random Walk Approach to First-Order Stochastic Convex Optimization

Cited by 10 publications

References 12 publications

Scalable Thompson Sampling using Sparse Gaussian Process Models

Scalable Thompson Sampling using Sparse Gaussian Process Models

Composite Anomaly Detection via Hierarchical Dynamic Search

Stochastic Gradient Descent on a Tree: an Adaptive and Robust Approach to Stochastic Convex Optimization

Contact Info

Product

Resources

About