Deep Learning for Click-Through Rate Estimation

Zhang, Weinan; Qin, Jiarui; Guo, Wei; Tang, Ruiming; He, Xiuqiang

doi:10.24963/ijcai.2021/636

Cited by 62 publications

(20 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is by no means to enumerate all studies on interaction modules but provide representative examples. A more comprehensive review can be found in [45,49]. Although multiple aforementioned studies have shown the benefits of high-order interaction in prediction accuracy, a more in-depth look reveals that those high-order interactions are not equally important.…”

Section: Related Work and Discussionmentioning

confidence: 99%

DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction

Zhang¹,

Luo²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning feature interactions is important to the model performance of online advertising services. As a result, extensive efforts have been devoted to designing effective architectures to learn feature interactions. However, we observe that the practical performance of those designs can vary from dataset to dataset, even when the order of interactions claimed to be captured is the same. That indicates different designs may have different advantages and the interactions captured by them have non-overlapping information. Motivated by this observation, we propose DHEN -a deep and hierarchical ensemble architecture that can leverage strengths of heterogeneous interaction modules and learn a hierarchy of the interactions under different orders. To overcome the challenge brought by DHEN's deeper and multi-layer structure in training, we propose a novel co-designed training system that can further improve the training efficiency of DHEN. Experiments of DHEN on large-scale dataset from CTR prediction tasks attained 0.27% improvement on the Normalized Entropy (NE) of prediction and 1.2x better training throughput than state-of-the-art baseline, demonstrating their effectiveness in practice.

show abstract

Section: Related Work and Discussionmentioning

confidence: 99%

DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction

Zhang¹,

Luo²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…where 𝜎 is the activation function. Here we take a deep CTR model with MLP layer as an example, and other deep CTR models can also be easily extended since most of them can be regarded as a variety of MLP with a set of weight matrices [35] However, as introduced in Section 1, directly generating this weight matrix has two challenges: (1) Time-and memory-efficiency. Assuming that APG adopts a single perceptron layer in Eq 3, the generation needs O (𝑁 𝑀𝐷) computation cost and O (𝑁 𝑀𝐷) memory cost.…”

Section: Re-parameterizationmentioning

confidence: 99%

“…For simplicity, in this paper, we mainly discuss the weight matrix 𝑊 . Our method can also be easily applied to the parameters of other modules (e.g., transformer, attention network, etc) in deep CTR models since most of them can be regarded as a variety of MLP with a set of weight matrices[35].…”

mentioning

confidence: 99%

APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction

Yan¹,

Wang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

In many web applications, deep learning-based CTR prediction models (deep CTR models for short) are widely adopted. Traditional deep CTR models learn patterns in a static manner, i.e., the network parameters are the same across all the instances. However, such a manner can hardly characterize each of the instances which may have different underlying distribution. It actually limits the representation power of deep CTR models, leading to sub-optimal results. In this paper, we propose an efficient, effective, and universal module, Adaptive Parameter Generation network (APG), where the parameters of deep CTR models are dynamically generated on-the-fly based on different instances. Extensive experimental evaluation results show that APG can be applied to a variety of deep CTR models and significantly improve their performance. We have deployed APG in the Taobao sponsored search system and achieved 3% CTR gain and 1% RPM gain respectively 1 .

show abstract

“…According to different model architectures, these models can be divided into two categories: feature interaction based models and user interest modeling based models. In the following, we give a brief introduction about these two kinds of models, interested readers can refer to the recent survey paper [26] for more details.…”

Section: A Deep Ctr Modelsmentioning

confidence: 99%

MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction

Guo¹,

Zhang²,

He³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

CTR prediction is essential for modern recommender systems. Ranging from early factorization machines to deep learning based models in recent years, existing CTR methods focus on capturing useful feature interactions or mining important behavior patterns. Despite the effectiveness, we argue that these methods suffer from the risk of label sparsity (i.e., the user-item interactions are highly sparse with respect to the feature space), label noise (i.e., the collected user-item interactions are usually noisy), and the underuse of domain knowledge (i.e., the pairwise correlations between samples). To address these challenging problems, we propose a novel Multi-Interest Self-Supervised learning (MISS) framework which enhances the feature embeddings with interest-level self-supervision signals.With the help of two novel CNN-based multi-interest extractors, self-supervision signals are discovered with full considerations of different interest representations (point-wise and union-wise), interest dependencies (short-range and long-range), and interest correlations (inter-item and intra-item). Based on that, contrastive learning losses are further applied to the augmented views of interest representations, which effectively improves the feature representation learning. Furthermore, our proposed MISS framework can be used as an "plug-in" component with existing CTR prediction models and further boost their performances. Extensive experiments on three large-scale datasets show that MISS significantly outperforms the state-of-the-art models, by up to 13.55% in AUC, and also enjoys good compatibility with representative deep CTR models.

show abstract

Deep Learning for Click-Through Rate Estimation

Cited by 62 publications

References 7 publications

DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction

DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction

APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction

MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction

Contact Info

Product

Resources

About