Structure-Enhanced Heterogeneous Graph Contrastive Learning

Zhu, Yanqiao; Xu, Yuehang; Cui, Hejie; Yang, Carl; Liu, Qiang; Wu, Shu

doi:10.1137/1.9781611977172.10

Cited by 26 publications

(20 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate the performance of our model against various baselines from shallow graph representation learning algorithms, including DeepWalk 37 , Metapath2vec 38 , HIN2vec 39 , HERec 40 , to GCL methods (e.g., DGI 41 , GRACE 16 , DMGI 20 , STENCIL 22 , HeCo 21 ) to supervised GNNs, like GCN 42 , GAT 10 , HAN 36 . Note that DMGI, STENCIL, HeCo are three GCL models dedicated for HIN.…”

Section: Methodsmentioning

confidence: 99%

“…However, these methods do not consider the sampling bias inherent in GCL, inevitably leading to sub-optimality. STENCIL 22 and HeCo 21 treat the aggregation of metapath-induced subgraphs as a novel view, and propose to apply metapath similarity to measure the hardness between nodes to synthesize hard negatives. However, they still fail to model the consistency and complementarity between metapath views.…”

Section: Related Workmentioning

confidence: 99%

“…For HIN, the best choice of pretext tasks is still unclear and each work presents its own solution. For example, Park et al 20 proposes to use metapaths to learn a shared consensus vector as node representation, Wang et al 21 performs contrast between the aggregation of metapaths (view 1) and network schema (view 2), and Zhu et al 22 iteratively maximize the mutual information between a single metapath and the aggregation of them. Despite these approaches attempting to incorporate the universal knowledge across all metapaths, we argue that they actually assume metapaths are independent (different from the complementary nature), failing to capture the consistency between metapaths and thus leading to sub-optimality.…”

mentioning

confidence: 99%

See 2 more Smart Citations

A multi-view contrastive learning for heterogeneous network embedding

Chen

Fang

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Graph contrastive learning has been developed to learn discriminative node representations on homogeneous graphs. However, it is not clear how to augment the heterogeneous graphs without substantially altering the underlying semantics or how to design appropriate pretext tasks to fully capture the rich semantics preserved in heterogeneous information networks (HINs). Moreover, early investigations demonstrate that contrastive learning suffer from sampling bias, whereas conventional debiasing techniques (e.g., hard negative mining) are empirically shown to be inadequate for graph contrastive learning. How to mitigate the sampling bias on heterogeneous graphs is another important yet neglected problem. To address the aforementioned challenges, we propose a novel multi-view heterogeneous graph contrastive learning framework in this paper. We use metapaths, each of which depicts a complementary element of HINs, as the augmentation to generate multiple subgraphs (i.e., multi-views), and propose a novel pretext task to maximize the coherence between each pair of metapath-induced views. Furthermore, we employ a positive sampling strategy to explicitly select hard positives by jointly considering semantics and structures preserved on each metapath view to alleviate the sampling bias. Extensive experiments demonstrate MCL consistently outperforms state-of-the-art baselines on five real-world benchmark datasets and even its supervised counterparts in some settings.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

A multi-view contrastive learning for heterogeneous network embedding

Chen

Fang

et al. 2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…where 𝑑 − 𝑖 ∈ D − 𝑖 are the in-batch negatives, and 𝜏 = 0.01 is the parameter for temperature. Contrastive pretraining improves both the alignment and uniformity for embeddings of sequences [14,21,27,28,39], which can better support the retrieval task. □ Document Retrieval using Label Names.…”

Section: Stage-i: Dense Retrieval With Label Namesmentioning

confidence: 99%

Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training

et al. 2023

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Scientific document classification is a critical task for a wide range of applications, but the cost of obtaining massive amounts of humanlabeled data can be prohibitive. To address this challenge, we propose a weakly-supervised approach for scientific document classification using label names only. In scientific domains, label names often include domain-specific concepts that may not appear in the document corpus, making it difficult to match labels and documents precisely. To tackle this issue, we propose WanDeR, which leverages dense retrieval to perform matching in the embedding space to capture the semantics of label names. We further design the label name expansion module to enrich the label name representations. Lastly, a self-training step is used to refine the predictions. The experiments on three datasets show that WanDeR outperforms the best baseline by 11.9% on average. Our code will be published at https://github.com/ritaranx/wander.

show abstract

“…Graph-based Negative Sampling Methods. Existing graph-based methods for negative sampling mostly concentrate on collaborative filtering (CF)-based recommendations [6,24], graph contrastive learning [21,23,28,28]. MixGCF [6] efficiently injects information of positive samples into negative samples via a mix-up mechanism.…”

Section: Related Workmentioning

confidence: 99%

Neighborhood-based Hard Negative Mining for Sequential Recommendation

Liu

Zhang

et al. 2023

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Negative sampling plays a crucial role in training successful sequential recommendation models. Instead of merely employing random negative sample selection, numerous strategies have been proposed to mine informative negative samples to enhance training and performance. However, few of these approaches utilize structural information. In this work, we observe that as training progresses, the distributions of node-pair similarities in different groups with varying degrees of neighborhood overlap change significantly, suggesting that item pairs in distinct groups may possess different negative relationships. Motivated by this observation, we propose a graph-based negative sampling approach based on neighborhood overlap (GNNO) to exploit structural information hidden in user behaviors for negative mining. GNNO first constructs a global weighted item transition graph using training sequences. Subsequently, it mines hard negative samples based on the degree of overlap with the target item on the graph. Furthermore, GNNO employs curriculum learning to control the hardness of negative samples, progressing from easy to difficult. Extensive experiments on three Amazon benchmarks demonstrate GNNO's effectiveness in consistently enhancing the performance of various state-of-theart models and surpassing existing negative sampling strategies. The code will be released at https://github.com/floatSDSDS/GNNO.

show abstract

Structure-Enhanced Heterogeneous Graph Contrastive Learning

Cited by 26 publications

References 0 publications

A multi-view contrastive learning for heterogeneous network embedding

A multi-view contrastive learning for heterogeneous network embedding

Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training

Neighborhood-based Hard Negative Mining for Sequential Recommendation

Contact Info

Product

Resources

About