Bandits With Heavy Tail

Bubeck, Sébastien; Cesa-Bianchi, Nicolò; Lugosi, Gábor

doi:10.1109/tit.2013.2277869

Cited by 219 publications

(264 citation statements)

References 18 publications

Supporting

Mentioning

255

Contrasting

Order By: Relevance

“…We conduct simulations with the following benchmarks -(i) an ε-greedy agent with linearly decreasing ε, (ii) Regular TS with Gaussian priors and a Gaussian assumption on the data (Gaussian-TS), (iii) Robust-UCB [Bubeck et al, 2013] for heavy-tailed distributions using a truncated mean estimator, and (iv) α-TS and (v) Robust α-TS, both with Q(iterations for sampling) as 50.…”

Section: Methodsmentioning

confidence: 99%

“…A version of the UCB algorithm [Auer et al, 2002] has been proposed in Bubeck et al [2013] coupled with several robust mean estimators to obtain Robust-UCB algorithms with optimal problem-dependent (i.e. dependent on individual µ k s) regret when rewards are heavy-tailed.…”

Section: Related Workmentioning

confidence: 99%

“…The most prominently studied class of algorithms are the Upper Confidence Bound (UCB) algorithms, that employ an "optimism in the face of uncertainty" heuristic Auer et al [2002], which have been shown to be optimal (in terms of regret) in several cases Cappé et al [2013]; Bubeck et al [2013]. Over the past few years, however, there has been a resurgence in interest in the Thompson Sampling (TS) algorithm Thompson [1933], that approaches the problem from a Bayesian perspective.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Thompson Sampling on Symmetric Alpha-Stable Bandits

Dubey

Pentland

2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Thompson Sampling provides an efficient technique to introduce prior knowledge in the multiarmed bandit problem, along with providing remarkable empirical performance. In this paper, we revisit the Thompson Sampling algorithm under rewards drawn from symmetric α-stable distributions, which are a class of heavy-tailed probability distributions utilized in finance and economics, in problems such as modeling stock prices and human behavior. We present an efficient framework for posterior inference, which leads to two algorithms for Thompson Sampling in this setting. We prove finite-time regret bounds for both algorithms, and demonstrate through a series of experiments the stronger performance of Thompson Sampling in this setting. With our results, we provide an exposition of symmetric α-stable distributions in sequential decision-making, and enable sequential Bayesian inference in applications from diverse fields in finance and complex systems that operate on heavy-tailed features.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Thompson Sampling on Symmetric Alpha-Stable Bandits

Dubey

Pentland

2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…Theorem 3.7 shows that such constant effort is inevitable for all (ε, δ)-approximating algorithms, with a lower bound that reproduces the dependence on K and δ. The cost bound (5) also reveals the adaption of our algorithm to the input, the lower bounds of Theorem 3.9 and 3.10 exhibit the same dependence on ε, δ, and the norm. In doing so, we obtain (up to constants) tight upper and lower bounds for the (ε, δ)-complexity of computing expected values (2).…”

Section: Introductionmentioning

confidence: 64%

“…Irrespective of the specified accuracy ε and the input random variable, the cost bound (5) is at least some constant times K pq/(q−p) log δ −1 . This fixed cost comes from estimating Y −E Y 1 within our particular algorithm and can be interpreted as the price we need to pay for not knowing the statistical dispersion of Y .…”

Section: Introductionmentioning

confidence: 99%

Solvable integration problems and optimal sample size selection

Kunsch

Novak

Rudolf³

2019

Journal of Complexity

View full text Add to dashboard Cite

We compute the integral of a function or the expectation of a random variable with minimal cost and use, for our new algorithm and for upper bounds of the complexity, i.i.d. samples. Under certain assumptions it is possible to select a sample size based on a variance estimation, or -more generally -based on an estimation of a (central absolute) p-moment. That way one can guarantee a small absolute error with high probability, the problem is thus called solvable. The expected cost of the method depends on the p-moment of the random variable, which can be arbitrarily large.In order to prove the optimality of our algorithm we also provide lower bounds. These bounds apply not only to methods based on i.i.d. samples but also to general randomized algorithms. They show that -up to constantsthe cost of the algorithm is optimal in terms of accuracy, confidence level, and norm of the particular input random variable. Since the considered classes of random variables or integrands are very large, the worst case cost would be infinite. Nevertheless one can define adaptive stopping rules such that for each input the expected cost is finite.We contrast these positive results with examples of integration problems that are not solvable.

show abstract

Robust covariance estimation for high‐dimensional compositional data with application to microbial communities analysis

Liu

Zhang

et al. 2021

Statistics in Medicine

View full text Add to dashboard Cite

Microbial communities analysis is drawing growing attention due to the rapid development fire of high‐throughput sequencing techniques nowadays. The observed data has the following typical characteristics: it is high‐dimensional, compositional (lying in a simplex) and even would be leptokurtic and highly skewed due to the existence of overly abundant taxa, which makes the conventional correlation analysis infeasible to study the co‐occurrence and co‐exclusion relationship between microbial taxa. In this article, we address the challenges of covariance estimation for this kind of data. Assuming the basis covariance matrix lying in a well‐recognized class of sparse covariance matrices, we adopt a proxy matrix known as centered log‐ratio covariance matrix in the literature. We construct a Median‐of‐Means estimator for the centered log‐ratio covariance matrix and propose a thresholding procedure that is adaptive to the variability of individual entries. By imposing a much weaker finite fourth moment condition compared with the sub‐Gaussianity condition in the literature, we derive the optimal rate of convergence under the spectral norm. In addition, we also provide theoretical guarantee on support recovery. The adaptive thresholding procedure of the MOM estimator is easy to implement and gains robustness when outliers or heavy‐tailedness exist. Thorough simulation studies are conducted to show the advantages of the proposed procedure over some state‐of‐the‐arts methods. At last, we apply the proposed method to analyze a microbiome dataset in human gut.

show abstract

Bandits With Heavy Tail

Cited by 219 publications

References 18 publications

Thompson Sampling on Symmetric Alpha-Stable Bandits

Thompson Sampling on Symmetric Alpha-Stable Bandits

Solvable integration problems and optimal sample size selection

Robust covariance estimation for high‐dimensional compositional data with application to microbial communities analysis

Contact Info

Product

Resources

About