Interpretable Click-Through Rate Prediction through Hierarchical Attention

Li, Zeyu; Cheng, Wei; Yang, Chen; Chen, Haifeng; Wang, Wei

doi:10.1145/3336191.3371785

Cited by 92 publications

(52 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, LE and RMSE straightforwardly evaluate the distance between the predicted possibility and the true label for each instance. It is worth noting that, in the context of CTR prediction where service providers commonly have a large user base, an improvement on the aforementioned metrics at the 0.001-level is regarded significant [15], [22], [45], [56].…”

Section: Evaluation Protocolsmentioning

confidence: 99%

Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling

Chen¹,

Yin²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Due to the sparsity of available features in web-scale predictive analytics, combinatorial features become a crucial means for deriving accurate predictions. As a well-established approach, factorization machine (FM) is capable of automatically learning high-order interactions among features to make predictions without the need for manual feature engineering. With the prominent development of deep neural networks (DNNs), there is a recent and ongoing trend of enhancing the expressiveness of FM-based models with DNNs. However, though better results are obtained with DNN-based FM variants, such performance gain is paid off by an enormous amount (usually millions) of excessive model parameters on top of the plain FM. Consequently, the heavy parameterization impedes the real-life practicality of those deep models, especially efficient deployment on resourceconstrained IoT and edge devices.In this paper, we move beyond the traditional real space where most deep FM-based models are defined, and seek solutions from quaternion representations within the hypercomplex space. Specifically, we propose the quaternion factorization machine (QFM) and quaternion neural factorization machine (QNFM), which are two novel lightweight and memory-efficient quaternion-valued models for sparse predictive analytics. By introducing a brand new take on FM-based models with the notion of quaternion algebra, our models not only enable expressive inter-component feature interactions, but also significantly reduce the parameter size due to lower degrees of freedom in the hypercomplex Hamilton product compared with real-valued matrix multiplication. Extensive experimental results on three large-scale datasets demonstrate that QFM achieves 4.36% performance improvement over the plain FM without introducing any extra parameters, while QNFM outperforms all baselines with up to two magnitudes' parameter size reduction in comparison to stateof-the-art peer methods.

show abstract

Section: Evaluation Protocolsmentioning

confidence: 99%

Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling

Chen¹,

Yin²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the design of handcrafted features used in these methods usually requires massive domain knowledge, and handcrafted features may not be optimal in modeling user interest. There are also several methods for display ads CTR prediction that use deep learning techniques to learn user interest representations from their behaviors on e-commerce platforms [9,18,31]. For example, Zhou et al [45] proposed a deep interest network (DIN) that learns representations of users from the items they have interacted with on the e-commerce platform based on the relatedness between those items and the candidate ads.…”

Section: Related Work 21 Ctr Predictionmentioning

confidence: 99%

FedCTR: Federated Native Ad CTR Prediction with Cross-platform User Behavior Data

Lyu³

et al. 2022

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

Native ad is a popular type of online advertisement which has similar forms with the native content displayed on websites. Native ad click-through rate (CTR) prediction is useful for improving user experience and platform revenue. However, it is challenging due to the lack of explicit user intent, and user behaviors on the platform with native ads may be insufficient to infer users’ interest in ads. Fortunately, user behaviors exist on many online platforms which can provide complementary information for user interest mining. Thus, leveraging multi-platform user behaviors is useful for native ad CTR prediction. However, user behaviors are highly privacy-sensitive, and the behavior data on different platforms cannot be directly aggregated due to user privacy concerns and data protection regulations. Existing CTR prediction methods usually require centralized storage of user behavior data for user modeling, which cannot be directly applied to the CTR prediction task with multi-platform user behaviors. In this paper, we propose a federated native ad CTR prediction method named FedCTR, which can learn user interest representations from cross-platform user behaviors in a privacy-preserving way. On each platform a local user model learns user embeddings from the local user behaviors on that platform. The local user embeddings from different platforms are uploaded to a server for aggregation, and the aggregated ones are sent to the ad platform for CTR prediction. Besides, we apply local differential privacy (LDP) and differential privacy (DP) to the local and aggregated user embeddings respectively for better privacy protection. Moreover, we propose a federated framework for collaborative model training with distributed models and user behaviors. Extensive experiments on real-world dataset show that FedCTR can effectively leverage multi-platform user behaviors for native ad CTR prediction in a privacy-preserving manner.

show abstract

“…For numerical features, we first divide the features into different numerical buckets. Then we apply one-hot encoding to identify different buckets, which is the similar way with [16]. Note that the one-hot encoding vectors can be extremely sparse because there are billions of item ids.…”

Section: Embedding Layermentioning

confidence: 99%

End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

Chen,

Pei,

et al. 2021

Preprint

View full text Add to dashboard Cite

Click-Through Rate (CTR) prediction is one of the core tasks in recommender systems (RS). It predicts a personalized click probability for each user-item pair. Recently, researchers have found that the performance of CTR model can be improved greatly by taking user behavior sequence into consideration, especially long-term user behavior sequence. The report on an e-commerce website shows that 23% of users have more than 1000 clicks during the past 5 months. Though there are numerous works focus on modeling sequential user behaviors, few works can handle long-term user behavior sequence due to the strict inference time constraint in real world system. Two-stage methods are proposed to push the limit for better performance. At the first stage, an auxiliary task is designed to retrieve the top-𝑘 similar items from long-term user behavior sequence. At the second stage, the classical attention mechanism is conducted between the candidate item and 𝑘 items selected in the first stage. However, information gap happens between retrieval stage and the main CTR task. This goal divergence can greatly diminishing the performance gain of long-term user sequence. In this paper, inspired by Reformer, we propose a localitysensitive hashing (LSH) method called ETA (End-to-end Target Attention) which can greatly reduce the training and inference cost and make the end-to-end training with long-term user behavior sequence possible. Both offline and online experiments confirm the effectiveness of our model. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.

show abstract

Interpretable Click-Through Rate Prediction through Hierarchical Attention

Cited by 92 publications

References 25 publications

Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling

Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling

FedCTR: Federated Native Ad CTR Prediction with Cross-platform User Behavior Data

End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

Contact Info

Product

Resources

About