Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation

Abstract

Accurate user and item embedding learning is crucial for modern recommender systems. However, most existing recommendation techniques have thus far focused on modeling users' preferences over singular type of user-item interactions. Many practical recommendation scenarios involve multi-typed user interactive behaviors (e.g., page view, add-to-favorite and purchase), which presents unique challenges that cannot be handled by current recommendation solutions. In particular: i) complex inter-dependencies across different types of user behaviors; ii) the incorporation of knowledge-aware item relations into the multi-behavior recommendation framework; iii) dynamic characteristics of multi-typed user-item interactions. To tackle these challenges, this work proposes a Knowledge-Enhanced Hierarchical Graph Transformer Network (KHGT), to investigate multi-typed interactive patterns between users and items in recommender systems. Specifically, KHGT is build upon a graph-structured neural architecture to i) capture type-specific behavior semantics; ii) explicitly discriminate which types of user-item interactions are more important in assisting the forecasting task on the target behavior. Additionally, we further integrate the multi-modal graph attention layer with temporal encoding strategy, to empower the learned embeddings be reflective of both dedicated multiplex user-item and item-item collaborative relations, as well as the underlying interaction dynamics. Extensive experiments conducted on three real-world datasets show that KHGT consistently outperforms many state-of-the-art recommendation methods across various evaluation settings. Our implementation is available in https://github.com/akaxlh/KHGT.

Coarse-to-Fine Knowledge-Enhanced Multi-Interest Learning Framework for Multi-Behavior Recommendation

“…Further, the GNN-based methods like MGNN [49], MBGCN [19], GHCF [9] and MBGMN [44] propose to leverage message passing on graphs to model high-order multi-behavioral interactive information. Moreover, KHGT [43] combines GNN and transformer together to model the global behavioral information, which not only captures the higher-order behavior between nodes, but also addresses the dynamics of behavior. The other category is to model different behaviors with MTL.…”

“…Knowledge-aware item-item relations are widely used to supplement semantic information and assist representation learning [5,39,43]. Inspired by the strong semantics of relations in the knowledgeaware relation graph [37,39,43], we propose a Coarse-grained Interest Extraction (CIE) module to extract users' interests which motivates users' interactions of multiple behaviors. In this way, we obtain the initial interest clustering centers.…”

“…where 𝑙 ∈ [0, 1, ..., 𝐿 𝑢−𝑖 ], 𝐿 𝑢−𝑖 is the number of GNN layer, (∥) is the concatenated operation for two vectors. To aggregate the information of each layer, we follow KHGT [43], and simply add them up. The unified formulation is:…”

“…where 𝑠 ∈ [1, 2, ..., 𝑁 * ] denotes the 𝑠-𝑡ℎ interest, (•) is the hadamard product operation. Finally, to perform the model optimization, we follow KHGT [43] and use marginal pair-wise Bayesian Personalized Ranking (BPR) loss function to minimize the following loss function:…”

“…where Y 𝑘 𝑢−𝑖 is the user-item adjacency interaction matrix of behavior 𝑘, A 𝑘 𝑢−𝑖 ∈ R (𝑀+𝑁 )×(𝑀+𝑁 ) , 𝑀 and 𝑁 denote the number of users and items, respectively. As for the processing of time, we simply follow KHGT [43], and initialize the embedding of time as…”