Adversarial Gradient Driven Exploration for Deep Click-Through Rate Prediction

Abstract

Nowadays, data-driven deep neural models have already shown remarkable progress on Click-through Rate (CTR) prediction. Unfortunately, the effectiveness of such models may fail when there are insufficient data. To handle this issue, researchers often adopt exploration strategies to examine items based on the estimated reward, e.g., UCB or Thompson Sampling. In the context of Exploitationand-Exploration for CTR prediction, recent studies have attempted to utilize the prediction uncertainty along with model prediction as the reward score. However, we argue that such an approach may make the final ranking score deviate from the original distribution, and thereby affect model performance in the online system. In this paper, we propose a novel exploration method called Adversarial Gradient Driven Exploration (AGE). Specifically, we propose a Pseudo-Exploration Module to simulate the gradient updating process, which can approximate the influence of the samples of to-be-explored items for the model. In addition, for better exploration efficiency, we propose an Dynamic Threshold Unit to eliminate the effects of those samples with low potential CTR. The effectiveness of our approach was demonstrated on an open-access academic dataset. Meanwhile, AGE has also been deployed in a real-world display advertising platform and all online metrics have been significantly improved.

An Incremental Update Framework for Online Recommenders with Data-Driven Prior

“…Under the numerous numbers of users and items, user clicks are extremely sparse, which makes it difficult to accurately estimate user preference. Recent research [26] suggests that each item requires around 10,000 impressions for convergence. Limited impressions hinder robust estimation during incremental updates.…”