UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation

Abstract

With the recent success of graph convolutional networks (GCNs), they have been widely applied for recommendation, and achieved impressive performance gains. The core of GCNs lies in its message passing mechanism to aggregate neighborhood information. However, we observed that message passing largely slows down the convergence of GCNs during training, especially for large-scale recommender systems, which hinders their wide adoption. LightGCN makes an early attempt to simplify GCNs for collaborative filtering by omitting feature transformations and nonlinear activations. In this paper, we take one step further to propose an ultra-simplified formulation of GCNs (dubbed UltraGCN), which skips infinite layers of message passing for efficient recommendation. Instead of explicit message passing, UltraGCN resorts to directly approximate the limit of infinite-layer graph convolutions via a constraint loss. Meanwhile, UltraGCN allows for more appropriate edge weight assignments and flexible adjustment of the relative importances among different types of relationships. This finally yields a simple yet effective UltraGCN model, which is easy to implement and efficient to train. Experimental results on four benchmark datasets show that UltraGCN not only outperforms the state-of-the-art GCN models but also achieves more than 10x speedup over LightGCN.

Applying self-attention model to learn both Empirical Risk Minimization and Invariant Risk Minimization for multimedia recommendation

“…UltraGCN [18] is an improved Graph Convolutional Networks (GCN) algorithm. It has the following advantages:…”

“…Adaptive neighbor sampling: UltraGCN [18] can flexibly sample neighbors based on the neighbor status of different nodes, reducing computational and storage costs and improving the scalability and efficiency of the algorithm.…”

“…2. Scalability: UltraGCN [18] has good scalability and can easily handle large-scale, dense graph data with significant runtime efficiency and accuracy advantages. This paper uses UltraGCN [18] to implement IRM and ERM.…”

“…3. Attention mechanism: UltraGCN [18] uses attention mechanisms to weigh different nodes and features, better exploring the relationships and importance between nodes and improving algorithm accuracy and robustness. In the paper, we apply the attention mechanism to combine IRM with ERM to form a better representation model.…”

“…Based on these advantages, UltraGCN [18] has been widely applied in graph neural networks, achieving good performance on various tasks and datasets.…”