When can the two-armed bandit algorithm be trusted?

Lamberton, Damien; Pagès, Gilles; Tarrès, Pierre

doi:10.1214/105051604000000350

Cited by 29 publications

(70 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A challenging question would be to prove (or disprove) that this event has zero probability when A ď 1. This is reminiscent of the situation thoroughly analyzed for two-armed bandit problems in [29,30].…”

Section: Examples and Applicationsmentioning

confidence: 92%

Stochastic approximation of quasi-stationary distributions on compact spaces and applications

Benaı̈m¹,

Cloez²,

Panloup³

2018

Ann. Appl. Probab.

View full text Add to dashboard Cite

As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a non-irreducible setting.

show abstract

Section: Examples and Applicationsmentioning

confidence: 92%

Stochastic approximation of quasi-stationary distributions on compact spaces and applications

Benaı̈m¹,

Cloez²,

Panloup³

2018

Ann. Appl. Probab.

View full text Add to dashboard Cite

show abstract

“…The optimal bids plotted in Figure 2 are consistent with intuition: the higher the remaining budget and the closer to the terminal time, the higher the optimal bid. alternative is to use multi-armed bandit algorithms such as those proposed in [18,19,20]. 22 A Pareto distribution or an extreme-value distribution are also relevant choices.…”

Section: Numerical Approximations Of V and B *mentioning

confidence: 99%

Optimal Real-Time Bidding Strategies

Fernandez-Tapia

Guéant

2016

Appl Math Res Express

View full text Add to dashboard Cite

The ad-trading desks of media-buying agencies are increasingly relying on complex algorithms for purchasing advertising inventory. In particular, Real-Time Bidding (RTB) algorithms respond to many auctions -usually Vickrey auctions -throughout the day for buying ad-inventory with the aim of maximizing one or several key performance indicators (KPI). The optimization problems faced by companies building bidding strategies are new and interesting for the community of applied mathematicians. In this article, we introduce a stochastic optimal control model that addresses the question of the optimal bidding strategy in various realistic contexts: the maximization of the inventory bought with a given amount of cash in the framework of audience strategies, the maximization of the number of conversions/acquisitions with a given amount of cash, etc. In our model, the sequence of auctions is modeled by a Poisson process and the price to beat for each auction is modeled by a random variable following almost any probability distribution. We show that the optimal bids are characterized by a Hamilton-Jacobi-Bellman equation, and that almost-closed-form solutions can be found by using a fluid limit. Numerical examples are also provided.

show abstract

“…This choice is justi¯ed by two main reasons, (i) the continuous problem is more natural to handle by using stochastic approximation techniques, and (ii) it joins the current literature on the topic, both from a stochastic-control perspective (Guéant et al, 2012; and from an online learning perspective (Laruelle et al, 2011;. Nevertheless, a natural alternative to our modeling, is to directly focus on the discrete grid of prices and de¯ne the optimal strategy by using a multi-armed bandit algorithm (Lamberton et al, 2004;Lamberton and Pag es, 2008). To our knowledge, this research direction has not been explored for the case of algorithmic-trading strategies.…”

Section: A Caveat Concerning Modeling Choicesmentioning

confidence: 99%