Simple Hierarchical PageRank Graph Neural Networks

Abstract

Graph neural networks (GNNs) have many variants for graph representation learning. Several works introduce PageRank into GNNs to improve its neighborhood aggregation capabilities. However, these methods leverage the general PageRank to perform complex neighborhood aggregation to obtain the final feature representation, which leads to high computational cost and oversmoothing. In this paper, we propose simple hierarchical PageRank graph neural networks (SHP-GNNs), which first utilize the simple PageRank to aggregate different neighborhood ranges of each node, and then leverage a jumping architecture to combine these aggregated features to enable hierarchical structure-aware representation. In this case, first, the simple PageRank turns the neighborhood aggregation process to no-learning, thereby reducing the computational complexity of the model. Then, the jumping structure combines the aggregation features of each node's different hierarchy (neighborhood range) to learn more informative feature representation. Finally, the successful combination of the above methods alleviate the oversmoothing problem of deep GNNs. Our experimental evaluation demonstrates that SHP-GNNs achieve or match state-of-the-art results in node classification tasks, text classification tasks, and community prediction tasks. Moreover, since SHP-GNNs' neighborhood aggregation is a no-learning process, SHP-GNNs are successfully extended to node clustering tasks.

A Missionary Training Program For University Students In South Korea

“…In this section, we introduce an advanced method for universal graph prompt tuning that is applied to the features of input nodes at the subgraph-level, referred to as Subgraph-level Universal Prompt Tuning (SUPT). Inspired by the concept of pixel-level prompts for images [1,4,23,27,34,44,52] and node-level prompts for graphs [7], SUPT similarly utilizes additional prompt features into the input space of graphs.…”

“…A key aspect of our work is to theoretically demonstrate that SUPT maintains universality under these conditions. Consequently, this implies that SUPT can also achieve the theoretical performance upper bound for any prompting function, as shown in the GPF paper [7]. In the context of a specified pre-training task 𝑡 from a set of tasks T, and given an input graph G characterized by its nodes 𝑋 and adjacency matrix 𝐴, we introduce a prompting function Ψ 𝑡 (•).…”

“…Subsequent research explores soft prompts [20,22,28], employing continuous vectors to surpass the limitations of natural language embeddings. In graph prompt tuning, studies have actively investigated the advantages [7,29,40], with a focus on soft prompts due to the unique nature of graph data. GPPT [40] introduces a "pre-train, prompt, fine-tune" framework to mitigate the objective gap using soft prompt, but it only deals with link prediction for pretraining and node classification for downstream task.…”

“…PRODIGY introduces a framework that introduces in-context learning on graph, making the model to be pre-trained to solve tasks across a wide range of tasks with prompt graph [14]. GPF and GPF-plus [7] add soft prompts to all node features of the input graph, enabling the method to be applied to any pre-training strategy. Despite these advancements, most methods face limitations in handling diverse pre-training strategies and downstream tasks.…”

“…However, when it comes to Graph Neural Networks (GNNs), the landscape of prompt tuning methods remains largely unexplored, particularly in terms of universal applicability [7]. Existing prompt tuning methods in GNNs are often tightly coupled with specific pre-training strategies, limiting their broader application [29,40].…”