Sub-Graph Contrast for Scalable Self-Supervised Graph Representation Learning

Jiao, Yizhu; Xiong, Yun; Zhang, Jiawei; Zhang, Yao; Zhang, Tianqi; Zhu, Yangyong

doi:10.1109/icdm50108.2020.00031

Cited by 91 publications

(56 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To this end, we propose to augment those short sequences with pseudo-prior items. Intuitively, we append a fabricated sub-sequence of items at the beginning of short sequences, which provide additional contexts [3,29]. To generate these pseudo-prior items, we pre-train a transformer from a reverse (i.e., right-to-left) direction of the original sequence to predict the prior item.…”

mentioning

confidence: 99%

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Liu

Fan

Wang

et al. 2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Sequential Recommendation characterizes the evolving patterns by modeling item sequences chronologically. The essential target of it is to capture the item transition correlations. The recent developments of transformer inspire the community to design effective sequence encoders, e.g., SASRec and BERT4Rec. However, we observe that these transformer-based models suffer from the cold-start issue, i.e., performing poorly for short sequences. Therefore, we propose to augment short sequences while still preserving original sequential correlations. We introduce a new framework for Augmenting Sequential Recommendation with Pseudo-prior items (ASReP). We firstly pre-train a transformer with sequences in a reverse direction to predict prior items. Then, we use this transformer to generate fabricated historical items at the beginning of short sequences. Finally, we fine-tune the transformer using these augmented sequences from the time order to predict the next item. Experiments on two real-world datasets verify the effectiveness of ASReP. The code is available on https://github.com/DyGRec/ASReP. CCS CONCEPTS• Information systems → Recommender systems; • Computing methodologies → Neural networks.

show abstract

mentioning

confidence: 99%

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Liu

Fan

Wang

et al. 2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…After 20 runs of training, we provide the mean classification accuracy on the test dataset of our approach, and we reuse the metrics from Kipf & Welling [1] for the performance of DeepWalk, as well as Label Propagation (LP) [9] and Planetoid [10]. We also reuse the metrics already in Jiao & Xiong [12] [11]. For all the there datasets, we set output dimension to 200 for our method.…”

Section: Node Classificationmentioning

confidence: 99%

Deep attributed network embedding by mutual information maximization

Tao

Feng

2021

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

There is a growing body of literature that recognizes the importance of network embedding. It intends to encode the graph structure information into a low-dimensional vector for each node in the graph, which benefits the downstream tasks. Most of recent works focus on supervised learning. But they are usually not feasible in real-world datasets owing to the high cost to obtain labels. To address this issue, we design a new unsupervised attributed network embedding method, deep attributed network embedding by mutual information maximization (DMIM). Our method focuses on maximizing mutual information between the hidden representations of the global topological structure and the node attributes, which allows us to obtain the node embedding without manual labeling. To illustrate the effectiveness of our method, we carry out the node classification task using the learned node embeddings. Compared with the state-of-the-art unsupervised methods, our method achieves superior results on various datasets.

show abstract

“…One type can be used for self-supervised learning. SUBG-CON [11] exploits the strong correlation between the central graph node and its sampled subgraphs to capture regional structure information. SUBG-CON is a self-supervised representation learning method based on subgraph contrast.…”

Section: Introductionmentioning

confidence: 99%

GraphMS: Drug Target Prediction Using Graph Representation Learning with Substructures

et al. 2021

View full text Add to dashboard Cite

The prediction of drug–target interactions is always a key task in the field of drug redirection. However, traditional methods of predicting drug–target interactions are either mediocre or rely heavily on data stacking. In this work, we proposed our model named GraphMS. We merged heterogeneous graph information and obtained effective node information and substructure information based on mutual information in graph embeddings. We then learned high quality representations for downstream tasks, and proposed an end–to–end auto–encoder model to complete the task of link prediction. Experimental results show that our method outperforms several state–of–the–art models. The model can achieve the area under the receiver operating characteristics (AUROC) curve of 0.959 and area under the precise recall curve (AUPR) of 0.847. We found that the mutual information between the substructure and graph–level representations contributes most to the mutual information index in a relatively sparse network. And the mutual information between the node–level and graph–level representations contributes most in a relatively dense network.

show abstract

Sub-Graph Contrast for Scalable Self-Supervised Graph Representation Learning

Cited by 91 publications

References 12 publications

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Deep attributed network embedding by mutual information maximization

GraphMS: Drug Target Prediction Using Graph Representation Learning with Substructures

Contact Info

Product

Resources

About