Dynamic Parameterized Network for CTR Prediction

Abstract

Learning to capture feature relations effectively and efficiently is essential in clickthrough rate (CTR) prediction of modern recommendation systems. Most existing CTR prediction methods model such relations either through tedious manuallydesigned low-order interactions or through inflexible and inefficient high-order interactions, which both require extra DNN modules for implicit interaction modeling. In this paper, we proposed a novel plug-in operation, Dynamic Parameterized Operation (DPO), to learn both explicit and implicit interaction instance-wisely. We showed that the introduction of DPO into DNN modules and Attention modules can respectively benefit two main tasks in CTR prediction, enhancing the adaptiveness of feature-based modeling and improving user behavior modeling with the instance-wise locality. Our Dynamic Parameterized Networks significantly outperforms state-of-the-art methods in the offline experiments on the public dataset and real-world production dataset, together with an online A/B test. Furthermore, the proposed Dynamic Parameterized Networks has been deployed in the ranking system of one of the world's largest e-commerce companies, serving the main traffic of hundreds of millions of active users.

Generalize for Future: Slow and Fast Trajectory Learning for CTR Prediction

“…Sharpness-aware minimization (SAM) (Foret et al 2020) However, the large computation complexity hinder their deployment on real-world application. CR (Zhu et al 2023) considers the difference between offline retraining and online serving and then develops the practical surrogate losses for maximizing the offline model performance towards online deployment. However, none of these works have addressed the problem of maximizing future performance.…”

“…Benefited from mitigation on distributional drift, the strategy of online fine-tuning with recent data can efficiently improve the generalization on the next serving stage (Zhu et al 2023;Rendle and Schmidt-Thieme 2008). In real-world setting, we only care about the future performance on test data D new .…”

“…Implementation Details. To evaluate our proposed method, we followed the experimental setup of (Zhu et al 2023) by conducting two various phase, i.e. one-pass continual learning and standard supervised training.…”

“…CTR prediction plays a crucial role in both research and industries, particularly in sectors such as recommendation systems and online advertising. Significant efforts have been made to develop better prediction models, with recent success achieved by deep neural networks due to their impressive capabilities in discovering personal behavior interests and complex feature interactions (Zhu et al 2020;Liu et al 2022a;Ivchenko et al 2022;Liu et al 2022b;Guo et al 2017;Song et al 2019;Zhu et al 2023;Liu et al 2023b;Zhu et al 2021;Liu et al 2023a). However, most of these studies primarily focus on the batch learning setting, assuming independent and identically distributed (i.i.d.…”

“…• Online Fine-tuning: In this stage, it requires online data(Rendle and Schmidt-Thieme 2008) or batch data(Zhu et al 2023) in a small time window ∆ (e.g. ∆=10 minutes) for serving next-time traffic.…”