GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks

Liu, Zemin; Yu, Xingtong; Zhang, Xin Ming

doi:10.1145/3543507.3583386

Cited by 38 publications

(31 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, Tan et al [196], [197] employ contrastive learning, by constructing a subgraph around a target node based on connectivity and generating three types of contrast pairs (node-node, node-subgraph, and subgraph-subgraph) for model training. Another work GraphPrompt [198] employs GNN prompting techniques, unifying pre-training and downstream tasks for effective knowledge transfer, achieving impressive results even with limited supervision.…”

Section: Few-shot Node Classificationmentioning

confidence: 99%

“…The primary challenge in few-shot node classification is to extract transferable knowledge from base classes to benefit novel classes. Recent studies in this area, as summarized in Notably, the adoption of innovative techniques like prompt tuning [198], generative models [71], and others could potentially enhance the representation of novel classes, indicating promising future directions. For a broader perspective, we suggest consulting the survey of few-shot learning on graphs [30] for additional insights.…”

Section: Fsnc On Hinsmentioning

confidence: 99%

“…Model pre-training GNN parameters transfer [195], contrastive learning [196], [197], prompting [198] Knowledge distillation GNNs to MLPs [57], KG models to MLPs [278], Random knowledge distillation [73] Data sharing…”

Section: Techniques Literaturementioning

confidence: 99%

See 2 more Smart Citations

The danger of dichloromethane in the synthesis and application of imidazole-based dyes

Liu

Chen

et al. 2022

Dyes and Pigments

View full text Add to dashboard Cite

Section: Few-shot Node Classificationmentioning

confidence: 99%

Section: Fsnc On Hinsmentioning

confidence: 99%

See 1 more Smart Citation

The danger of dichloromethane in the synthesis and application of imidazole-based dyes

Liu

Chen

et al. 2022

Dyes and Pigments

View full text Add to dashboard Cite

“…Subsequent research explores soft prompts [20,22,28], employing continuous vectors to surpass the limitations of natural language embeddings. In graph prompt tuning, studies have actively investigated the advantages [7,29,40], with a focus on soft prompts due to the unique nature of graph data. GPPT [40] introduces a "pre-train, prompt, fine-tune" framework to mitigate the objective gap using soft prompt, but it only deals with link prediction for pretraining and node classification for downstream task.…”

Section: Related Workmentioning

confidence: 99%

“…However, when it comes to Graph Neural Networks (GNNs), the landscape of prompt tuning methods remains largely unexplored, particularly in terms of universal applicability [7]. Existing prompt tuning methods in GNNs are often tightly coupled with specific pre-training strategies, limiting their broader application [29,40]. For example, a prompt tuning method designed for a GNN pretrained on node classification tasks may not be effective for a GNN pre-trained on link prediction tasks.…”

Section: Introductionmentioning

confidence: 99%

A Missionary Training Program For University Students In South Korea

Lee¹

View full text Add to dashboard Cite

Sequential recommendation addresses the issue of preference drift by predicting the next item based on the user's previous behaviors. Recently, a promising approach using contrastive learning has emerged, demonstrating its effectiveness in recommending items under sparse user-item interactions. Significantly, the effectiveness of combinations of various augmentation methods has been demonstrated in different domains, particularly in computer vision. However, when it comes to augmentation within a contrastive learning framework in sequential recommendation, previous research has only focused on limited conditions and simple structures. Thus, it is still possible to extend existing approaches to boost the effects of augmentation methods by using progressed structures with the combinations of multiple augmentation methods. In this work, we propose a novel framework called Hierarchical Contrastive Learning with Multiple Augmentation for Sequential Recommendation (HCLRec) to overcome the aforementioned limitation. Our framework leverages existing augmentation methods hierarchically to improve performance. By combining augmentation methods continuously, we generate low-level and high-level view pairs. We employ a Transformers-based model to encode the input sequence effectively. Furthermore, we introduce additional blocks consisting of Transformers and position-wise feed-forward network (PFFN) layers to learn the invariance of the original sequences from hierarchically augmented views. We pass the input sequence to subsequent layers based on the number of increment levels applied to the views to handle various augmentation levels. Within each layer, we compute contrastive loss between pairs of views at the same level. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art approaches and that HCLRec is robust even when faced with the problem of sparse interaction. Our implementation is available at the link: https://.

show abstract