Recurrent Neural Networks for Stochastic Control in Real-Time Bidding

Abstract

Bidding in real-time auctions can be a difficult stochastic control task; especially if underdelivery incurs strong penalties and the market is very uncertain.Most current works and implementations focus on optimally delivering a campaign given a reasonable forecast of the market. Practical implementations have a feedback loop to adjust and be robust to forecasting errors, but no implementation, to the best of our knowledge, uses a model of market risk and actively anticipates market shifts.Solving such stochastic control problems in practice is actually very challenging. This paper proposes an approximate solution based on a Recurrent Neural Network (RNN) architecture that is both effective and practical for implementation in a production environment. The RNN bidder provisions everything it needs to avoid missing its goal. It also deliberately falls short of its goal when buying the missing impressions would cost more than the penalty for not reaching it.

Optimal Real-time Bidding Policies for Contract Fulfillment in Second Price Auctions

“…x). Such discontinuities are observed in real data, see e.g., [17,12,23]). Discontinuities may arise even in estimated supply curves e.g.…”

“…Many other algorithms have been been deployed on the problem of optimal bidding, including classical feedback control in the work of [35,16] where they seek to track certain keep performance indicators and [4,12,31] which utilize the Markov Decision framework.…”

Causal Models for Real Time Bidding with Repeated User Interactions

“…Second, it is common for the bidder to have some constraints on the advertising campaign, such as a maximum budget per day, or a maximum cost per click [Conitzer et al, 2018, Heymann, 2019. While in several cases a linear scaling applied to the display valuation may be enough to optimally satisfy a budget constraint, the value of this scaling factor is not known in advance, and several articles propose to update the bidding strategy to better take such constraint into account [Cai et al, 2017, Grislain et al, 2019, Lee et al, 2013, Yang et al, 2019. In the present work, we do not consider such constraints, but those methods might be applied on top of ours.…”

“…The bidder should set its bid as a function of the valuation ( ) following auction theory [Krishna, 2009]. This bid adjustment in the specific context of RTB has been studied in [Cai et al, 2017, Despotakis et al, 2019, Grislain et al, 2019, Lee et al, 2013, Yang et al, 2019 and is not in the scope on this work. For example, in the simple setting of second price auction, the optimal bid is deterministic, * = ( ).…”

Prospective Modeling of Users for Online Display Advertising via Deep Time-Aware Model

“…For conversion predictions the bid is often controlled by the probability of user converting after ad is displayed, more precisely, the maximum bid is defined as a factored conversion probability [11]. Function f represents optimal bidding strategy (often a linear function 2 ) and α is often called a control parameter which includes several signals such as pacing [9].…”