Heterogeneous Graph Transformer

Hu, Ziniu; Dong, Yuxiao; Wang, Kuansan; Sun, Yizhou

doi:10.1145/3366423.3380027

Cited by 783 publications

(542 citation statements)

References 17 publications

Supporting

Mentioning

541

Contrasting

Unclassified

Order By: Relevance

“…Each paper is labeled with a set of research topics/fields (e.g., Physics and Medicine) and the publication date ranges from 1900 to 2019. We consider the prediction of Paper-Field, Paper-Venue, and Author Name Disambiguation (Author ND) as three downstream tasks [7,15]. The performance is evaluated by MRR-a widely adopted ranking metric [19].…”

Section: Methodsmentioning

confidence: 99%

“…The base GNN model. On the OAG and Amazon datasets, we use the state-of-the-art heterogeneous GNN-Heterogeneous Graph Transformer (HGT) [15]-as the base model for GPT-GNN. Furthermore, we also use other (heterogeneous) GNNs as the base model to test our generative pre-training framework.…”

Section: Methodsmentioning

confidence: 99%

“…Recent years have witnessed the success of GNNs for modeling graph data [12,15,17,36]. A GNN can be regarded as using the input graph structure as the computation graph for message passing [9], during which the local neighborhood information is aggregated to get a more contextual representation.…”

Section: Preliminaries Of Graph Neural Networkmentioning

confidence: 99%

“…For example, Velickovi et al [36] propose the graph attention network (GAT), which adopts an additive mechanism to calculate attention and uses the same weight for calculating messages. Recently, Hu et al propose the heterogeneous graph transformer (HGT) [15] that leverages multi-head attentions for different relation types to get type-dependent attentions. The proposed pre-training framework GPT-GNN can apply to all of these GNN models.…”

Section: Preliminaries Of Graph Neural Networkmentioning

confidence: 99%

“…To pre-train GNNs on graphs that are too large to fit into the hardware, we sample subgraphs for training. In particular, we propose to sample a dense subgraph from homogeneous and heterogeneous graphs by using the LADIES algorithm [45] and its heterogeneous version HGSampling [15], respectively. Both methods theoretically guarantee that the sampled nodes are highly interconnected with each other and maximally preserve the structural information.…”

Section: Gpt-gnn For Heterogeneous and Large Graphsmentioning

confidence: 99%

See 4 more Smart Citations

GPT-GNN

Dong

Wang

et al. 2020

Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

Self Cite

264

View full text Add to dashboard Cite

Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data. However, training GNNs usually requires abundant task-specific labeled data, which is often arduously expensive to obtain. One effective way to reduce the labeling effort is to pre-train an expressive GNN model on unlabeled data with self-supervision and then transfer the learned model to downstream tasks with only a few labels. In this paper, we present the GPT-GNN * framework to initialize GNNs by generative pre-training. GPT-GNN introduces a self-supervised attributed graph generation task to pre-train a GNN so that it can capture the structural and semantic properties of the graph. We factorize the likelihood of the graph generation into two components: 1) Attribute Generation and 2) Edge Generation. By modeling both components, GPT-GNN captures the inherent dependency between node attributes and graph structure during the generative process. Comprehensive experiments on the billion-scale Open Academic Graph and Amazon recommendation data demonstrate that GPT-GNN significantly outperforms state-of-the-art GNN models without pre-training by up to 9.1% across various downstream tasks. CCS CONCEPTS • Computing methodologies → Unsupervised learning; Neural networks; Learning latent representations.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Preliminaries Of Graph Neural Networkmentioning

confidence: 99%

Section: Preliminaries Of Graph Neural Networkmentioning

confidence: 99%

Section: Gpt-gnn For Heterogeneous and Large Graphsmentioning

confidence: 99%

See 3 more Smart Citations

GPT-GNN

Dong

Wang

et al. 2020

Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

Self Cite

264

View full text Add to dashboard Cite

show abstract

Heterogeneous graph prompt for Community Question Answering

Liu

Qin

2022

Concurrency and Computation

View full text Add to dashboard Cite

SummaryCompared with general question answering, Community Question Answering (CQA), which has been widely used in various scenarios like E‐commerce and is well welcomed. In order to answer the user's question precisely, many CQA models resort to external knowledge sources such as Wikipedia. The main challenge of the task is knowledge extraction and utilization. Different from the traditional method of designing task‐specific knowledge modules, we propose a graph prompt‐based learning method that directly steers the pretrained language model to solve CQA tasks. Multiple information sources are organized as graph prompts to guide the generation of the model, naturally leveraging the knowledge learned in the pretrained step. Based on the pretrained bidirectional and autoregressive transformers, a large‐scale language model, a comparable performance is achieved with less than 10% of the full‐finetuning time by only optimizing the graph prompt parameters. Experiments on two standard CQA datasets show that compared with traditional sequential initialized prompts, graph prompt achieves 20.47% and 14.89% increments in terms of BLEU and ROUGE‐L scores on quick finetuning and outperforms in few‐shot learning.

show abstract

Prediction of Clinical Trials Outcomes Based on Target Choice and Clinical Trial Design with Multi‐Modal Artificial Intelligence

Aliper¹,

Kudrin²,

Polykovskiy³

et al. 2023

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Drug discovery and development is a notoriously risky process with high failure rates at every stage, including disease modeling, target discovery, hit discovery, lead optimization, preclinical development, human safety, and efficacy studies. Accurate prediction of clinical trial outcomes may help significantly improve the efficiency of this process by prioritizing therapeutic programs that are more likely to succeed in clinical trials and ultimately benefit patients. Here, we describe inClinico, a transformer‐based artificial intelligence software platform designed to predict the outcome of phase II clinical trials. The platform combines an ensemble of clinical trial outcome prediction engines that leverage generative artificial intelligence and multimodal data, including omics, text, clinical trial design, and small molecule properties. inClinico was validated in retrospective, quasi‐prospective, and prospective validation studies internally and with pharmaceutical companies and financial institutions. The platform achieved 0.88 receiver operating characteristic area under the curve in predicting the phase II to phase III transition on a quasi‐prospective validation dataset. The first prospective predictions were made and placed on date‐stamped preprint servers in 2016. To validate our model in a real‐world setting, we published forecasted outcomes for several phase II clinical trials achieving 79% accuracy for the trials that have read out. We also present an investment application of inClinico using date stamped virtual trading portfolio demonstrating 35% 9‐month return on investment.

show abstract

Heterogeneous Graph Transformer

Cited by 783 publications

References 17 publications

GPT-GNN

GPT-GNN

Heterogeneous graph prompt for Community Question Answering

Prediction of Clinical Trials Outcomes Based on Target Choice and Clinical Trial Design with Multi‐Modal Artificial Intelligence

Contact Info

Product

Resources

About