Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising

Abstract

Click-through rate (CTR) prediction is a critical task in online display advertising. The data involved in CTR prediction are typically multi-field categorical data, i.e., every feature is categorical and belongs to one and only one field. One of the interesting characteristics of such data is that features from one field often interact differently with features from different other fields. Recently, Fieldaware Factorization Machines (FFMs) have been among the best performing models for CTR prediction by explicitly modeling such difference. However, the number of parameters in FFMs is in the order of feature number times field number, which is unacceptable in the real-world production systems. In this paper, we propose Field-weighted Factorization Machines (FwFMs) to model the different feature interactions between different fields in a much more memory-efficient way. Our experimental evaluations show that FwFMs can achieve competitive prediction performance with only as few as 4% parameters of FFMs. When using the same number of parameters, FwFMs can bring 0.92% and 0.47% AUC lift over FFMs on two real CTR prediction data sets.

Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising

“…• Lead: the user fills out an online form • View Content: the user views a web page such as the landing page or a product page • Purchase: the user purchases a product • Sign Up: the user signs up an account The decisive factors, i.e., the main effect terms (fields) and/or the interaction terms (field pairs) that drive a user to convert, may vary a lot among these types. Following the analysis in [24], we verify this by computing mutual information (MI) between each field pair and each type of conversion on our real-world data set described later in section 4.1. Suppose there are M unique features {x 1 , .…”

“…MT-FwFM is a variant of Field-weighted Factorization Machine (FwFM), which is introduced in [24] for click prediction. FwFM is formulated as…”

“…There has been lots of work in the literature on click and conversion prediction in online advertising. Research on click prediction focus on developing various models, including Logistic Regression (LR) [8,23,27], Polynomial-2 (Poly2) [6], tree-based models [16], tensor-based models [26], Bayesian models [13], Field-aware Factorization Machines (FFM) [18,19], and Field-weighted Factorization Machines (FwFM) [24]. Recently, deep learning for CTR prediction also attracted a lot of research attention [9,14,15,25,29,30,32].…”

“…Besides task-specific parameters, these models share low level feature representations, providing the benefit of information sharing among different conversion types. We propose Multi-Task Field-weighted Factorization Machine (MT-FwFM), based on one of the best-performing models for click prediction, i.e., Field-weighted Factorization Machine (FwFM) [24], to solve these tasks together.…”

Field-Regularised Factorization Machines for Mining the Maintenance Logs of Equipment

“…They are more complex than FMs in terms of the number of parameters and computational complexity. Field-weighted Factorization Machines [16] add additional coefficients to depict the interactions of fields, and reduce the number of model parameters compared to FFMs. These models treat features from different fields differently.…”