Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction

Abstract

Cross features play an important role in click-through rate (CTR) prediction. Most of the existing methods adopt a DNN-based model to capture the cross features in an implicit manner. These implicit methods may lead to a sub-optimized performance due to the limitation in explicit semantic modeling. Although traditional statistical explicit semantic cross features can address the problem in these implicit methods, such features still suffer from some challenges, including lack of generalization and expensive memory cost. Few works focus on tackling these challenges. In this paper, we take the first step in learning the explicit semantic cross features and propose Pre-trained Cross Feature learning Graph Neural Networks (PCF-GNN), a GNN based pre-trained model aiming at generating cross features in an explicit fashion. Extensive experiments are conducted on both public and industrial datasets, where PCF-GNN shows competence in both performance and memory-efficiency in various tasks. CCS CONCEPTS• Information systems → Recommender systems.

Causality-based CTR prediction using graph neural networks

“…Due to their strength in graph representations, GNNs have been used to alleviate the feature sparsity and behavior sparsity problems in CTR prediction, by converting feature interactions into node interactions in a graph structure. Li et al (2019b) proposed a feature interaction GNNs (Fi-GNN) where field-aware feature interactions were realized through assigning two interaction matrices to each node in a complete graph; in a later work, Li et al (2021a) used the pre-trained GNNs to generate explicit semantic-cross features and applied the weighted square loss to compute the importance of these features.…”

“…Recently, quite a few studies have been reported to establish CTR prediction models in the GNNs framework, by modeling high-order interactions in feature graph (Li et al, 2019b;Li et al, 2021a;Li et al, 2021b), designing graph intention networks (Li et al, 2019a) and dynamic sequential graph (Chu et al, 2021) to enrich users' behaviors, constructing attribution graph and collaborative graph to address the sparsity issue (Guo et al, 2021). However, most of existing GNNs-based models handle feature interactions by aggregating information from all neighbors equally in a complete graph (Tao et al, 2020).…”

Materials fatigue prediction using graph neural networks on microstructure representations

“…However, the message passing phase of both GCN and GIN (see Table 5) aggregates features from neighbors before computing any feature combinations, thus ignoring the possible interactions between features of grain pairs. In literature, different approaches to learning sophisticated feature interaction across adjacent nodes have been proposed [26][27][28] , which might be interesting to model microstructural deformation. The GCN performing slightly better than the GIN, see Table 1 might also be linked to the aggregation of messages across neighbor grains.…”

Neural Graph Matching for Pre-training Graph Neural Networks

“…Generally, existing designed supervised signals can be classified into two main categories. The first is called node-level tasks, which aims at predicting localized properties utilizing node representations, such as graph structure reconstruction [3,6,7,15,11], localized attribute prediction [6,7] and node representation recovery [4]. Another is called graph-level tasks, which defines globalized optimization goal for the entire graph, such as graph property prediction [6,12] and mutual information maximization [21,28,17,15,27].…”