An Asymptotically Optimal Policy for Uniform Bandits of Unknown Support

Cowan, Wesley; Katehakis, Michael N.

doi:10.48550/arxiv.1505.01918

Cited by 5 publications

(8 citation statements)

References 14 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, bounded bandits can have an arbitrarily high kurtosis, so our settings are not directly comparable (and we think that bounded distributions with an unknown range is a more natural assumption). Cowan and Katehakis [14] study adaptation to the range but in the restricted case of uniform distributions; see also similar results by Cowan et al [15] for Gaussian distributions with unknown means and variances. Additional important references are discussed in Appendix A of the supplementary material.…”

Section: Introductionsupporting

confidence: 68%

Adaptation to the Range in $K$-Armed Bandits

Hadiji¹,

Stoltz²

2020

Preprint

View full text Add to dashboard Cite

We consider stochastic bandit problems with K arms, each associated with a bounded distribution supported on the range [m, M ]. We do not assume that the range [m, M ] is known and show that there is a cost for learning this range. Indeed, a new trade-off between distribution-dependent and distribution-free regret bounds arises, which, for instance, prevents from simultaneously achieving the typical ln T and √T bounds. For instance, a √ T distribution-free regret bound may only be achieved if the distribution-dependent regret bounds are at least of order √T . We exhibit a strategy achieving the rates for regret indicated by the new trade-off.Preprint. Under review.

show abstract

Section: Introductionsupporting

confidence: 68%

Adaptation to the Range in $K$-Armed Bandits

Hadiji¹,

Stoltz²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…These policies form the basis for deriving logarithmic regret polices for more general models, cf. Auer et al (2002), Auer and Ortner (2010), Cowan et al (2015), Cowan and Katehakis (2015a).…”

Section: Introductionmentioning

confidence: 99%

Asymptotically Optimal Multi-Armed Bandit Policies Under a Cost Constraint

Burnetas¹,

Kanavetas

Katehakis

2016

Prob. Eng. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

We develop asymptotically optimal policies for the multi armed bandit (MAB), problem, under a cost constraint. This model is applicable in situations where each sample (or activation) from a population (bandit) incurs a known bandit dependent cost. Successive samples from each population are iid random variables with unknown distribution. The objective is to design a feasible policy for deciding from which population to sample from, so as to maximize the expected sum of outcomes of n total samples or equivalently to minimize the regret due to lack on information on sample distributions, For this problem we consider the class of feasible uniformly fast (f-UF) convergent policies, that satisfy the cost constraint sample-path wise. We first establish a necessary asymptotic lower bound for the rate of increase of the regret function of f-UF policies. Then we construct a class of f-UF policies and provide conditions under which they are asymptotically optimal within the class of f-UF policies, achieving this asymptotic lower bound. At the end we provide the explicit form of such policies for the case in which the unknown distributions are Normal with unknown means and known variances. . Asymptotic optimality, finite horizon regret bounds, and a solution to an open problem. optimal Bayesian sequential change detection and identification rules. -armed bandit with budget constraint and variable costs. In AAAI-13, pages 232-238, 2013.Eugene A Feinberg, Pavlo O Kasyanov, and Michael Z Zgurovsky. Convergence of value iterations for total-cost mdps and pomdps with general state and action sets.

show abstract

“…The infimum over the KL divergence can be explicitly computed for any i ̸ = 1 under uniform models (Cowan and Katehakis 2015) as…”

Section: Problem Formulationmentioning

confidence: 99%

The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models

Lee,

Chiang,

Sugiyama

2024

AAAI

View full text Add to dashboard Cite

Thompson sampling (TS) has been known for its outstanding empirical performance supported by theoretical guarantees across various reward models in the classical stochastic multi-armed bandit problems. Nonetheless, its optimality is often restricted to specific priors due to the common observation that TS is fairly insensitive to the choice of the prior when it comes to asymptotic regret bounds. However, when the model contains multiple parameters, the optimality of TS highly depends on the choice of priors, which casts doubt on the generalizability of previous findings to other models. To address this gap, this study explores the impact of selecting noninformative priors, offering insights into the performance of TS when dealing with new models that lack theoretical understanding. We first extend the regret analysis of TS to the model of uniform distributions with unknown supports, which would be the simplest non-regular model. Our findings reveal that changing noninformative priors can significantly affect the expected regret, aligning with previously known results in other multiparameter bandit models. Although the uniform prior is shown to be optimal, we highlight the inherent limitation of its optimality, which is limited to specific parameterizations and emphasizes the significance of the invariance property of priors. In light of this limitation, we propose a slightly modified TS-based policy, called TS with Truncation (TS-T), which can achieve the asymptotic optimality for the Gaussian models and the uniform models by using the reference prior and the Jeffreys prior that are invariant under one-to-one reparameterizations. This policy provides an alternative approach to achieving optimality by employing fine-tuned truncation, which would be much easier than hunting for optimal priors in practice.

show abstract

An Asymptotically Optimal Policy for Uniform Bandits of Unknown Support

Cited by 5 publications

References 14 publications

Adaptation to the Range in $K$-Armed Bandits

Adaptation to the Range in $K$-Armed Bandits

Asymptotically Optimal Multi-Armed Bandit Policies Under a Cost Constraint

The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models

Contact Info

Product

Resources

About