Factorization machines with follow-the-regularized-leader for CTR prediction in display advertising

Ta, Anh-Phuong

doi:10.1109/bigdata.2015.7364112

Cited by 42 publications

(24 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We take CTR estimation in online advertising as the working example to explore the learning ability of our PNN model. The extensive experimental results on two large-scale realworld datasets demonstrate the consistent superiority of our model over state-of-the-art user response prediction models [6], [13], [12] on various metrics.…”

Section: Introductionmentioning

confidence: 64%

“…The data collection in these IR tasks is mostly in a multifield categorical form, for example, [Weekday=Tuesday, Gender=Male, City=London], which is normally transformed into high-dimensional sparse binary features via onehot encoding [4]. For example, the three field vectors with one-hot encoding are concatenated as Many machine learning models, including linear logistic regression [5], non-linear gradient boosting decision trees [4] and factorization machines [6], have been proposed to work on such high-dimensional sparse binary features and produce high quality user response predictions. However, these models highly depend on feature engineering in order to capture highorder latent patterns [7].…”

Section: Introductionmentioning

confidence: 99%

“…More importantly, the "add" operations of the perceptron layer might not be useful to explore the interactions of categorical data in multiple fields. Previous work [1], [6] has shown that local dependencies between features from different fields can be effectively explored by feature vector "product" operations instead of "add" operations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Product-Based Neural Networks for User Response Prediction

Cai

Ren

et al. 2016

2016 IEEE 16th International Conference on Data Mining (ICDM)

581

468

View full text Add to dashboard Cite

Abstract-Predicting user responses, such as clicks and conversions, is of great importance and has found its usage in many Web applications including recommender systems, web search and online advertising. The data in those applications is mostly categorical and contains multiple fields; a typical representation is to transform it into a high-dimensional sparse binary feature representation via one-hot encoding. Facing with the extreme sparsity, traditional models may limit their capacity of mining shallow patterns from the data, i.e. low-order feature combinations. Deep models like deep neural networks, on the other hand, cannot be directly applied for the high-dimensional input because of the huge feature space. In this paper, we propose a Product-based Neural Networks (PNN) with an embedding layer to learn a distributed representation of the categorical data, a product layer to capture interactive patterns between interfield categories, and further fully connected layers to explore high-order feature interactions. Our experimental results on two large-scale real-world ad click datasets demonstrate that PNNs consistently outperform the state-of-the-art models on various metrics.

show abstract

Section: Introductionmentioning

confidence: 64%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Product-Based Neural Networks for User Response Prediction

Cai

Ren

et al. 2016

2016 IEEE 16th International Conference on Data Mining (ICDM)

581

468

View full text Add to dashboard Cite

show abstract

“…where to simplify our notation, we substitute f θ (x) with its predicted CTR variable r. (24) where µ is the Lagrangian multiplier. Taking the derivative equal to zero, we get that To solve µ, we take the Lagrangian derivative w.r.t.…”

Section: Joint Optimization With Bidding Functionmentioning

confidence: 99%

“…From the methodology view, linear models such as logistic regression [14] and non-linear models such as tree-based model [10] and factorization machines [19,21] are commonly used. Other variants include Bayesian probit regression [9], FTRFL [24] in factorization machine, and convolutional neural network learning framework [17]. Normally, area under ROC curve (AUC) and relative information gain (RIG) are common evaluation metrics for CTR prediction accuracy [9].…”

Section: Related Workmentioning

confidence: 99%

User Response Learning for Directly Optimizing Campaign Performance in Display Advertising

Ren

Zhang

Rong

et al. 2016

Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Learning and predicting user responses, such as clicks and conversions, are crucial for many Internet-based businesses including web search, e-commerce, and online advertising. Typically, a user response model is established by optimizing the prediction accuracy, e.g., minimizing the error between the prediction and the ground truth user response. However, in many practical cases, predicting user responses is only part of a rather larger predictive or optimization task, where on one hand, the accuracy of a user response prediction determines the final (expected) utility to be optimized, but on the other hand, its learning may also be influenced from the follow-up stochastic process. It is, thus, of great interest to optimize the entire process as a whole rather than treat them independently or sequentially. In this paper, we take real-time display advertising as an example, where the predicted user's ad click-through rate (CTR) is employed to calculate a bid for an ad impression in the second price auction. We reformulate a common logistic regression CTR model by putting it back into its subsequent bidding context: rather than minimizing the prediction error, the model parameters are learned directly by optimizing campaign profit. The gradient update resulted from our formulations naturally fine-tunes the cases where the market competition is high, leading to a more costeffective bidding. Our experiments demonstrate that, while maintaining comparable CTR prediction accuracy, our proposed user response learning leads to campaign profit gains as much as 78.2% for offline test and 25.5% for online A/B test over strong baselines.

show abstract

Machine‐Learning and Deep‐Learning Techniques in Social Sciences

Bhagat

Singh

2022

Machine Learning Algorithms for Signal and Image Processing

View full text Add to dashboard Cite

Factorization machines with follow-the-regularized-leader for CTR prediction in display advertising

Cited by 42 publications

References 4 publications

Product-Based Neural Networks for User Response Prediction

Product-Based Neural Networks for User Response Prediction

User Response Learning for Directly Optimizing Campaign Performance in Display Advertising

Machine‐Learning and Deep‐Learning Techniques in Social Sciences

Contact Info

Product

Resources

About