Cluster-GCN

Chiang, Wei-Lin; Liu, Xuanqing; Si, Si; Li, Yang; Bengio, Samy; Hsieh, Cho‐Jui

doi:10.1145/3292500.3330925

Cited by 705 publications

(112 citation statements)

References 9 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…Generally all deep learning models are trained by stochastic gradient descent which requires us to minibatch our graphs. To breakdown a large graph for memory purposes and fast efficient training, we follow the algorithm in [9] which is implemented in PyTorch Geometric. For the inductive task, we break up our training graph into 4000 roughly equal subgraphs and use a batch size of 256 graphs.…”

Section: Methodsmentioning

confidence: 99%

Disease state prediction from single-cell data using graph attention networks

Ravindra

Sehanobish

Pappalardo

et al. 2020

Proceedings of the ACM Conference on Health, Inference, and Learning

View full text Add to dashboard Cite

Single-cell RNA sequencing (scRNA-seq) has revolutionized biological discovery, providing an unbiased picture of cellular heterogeneity in tissues. While scRNA-seq has been used extensively to provide insight into both healthy systems and diseases, it has not been used for disease prediction or diagnostics. Graph Attention Networks (GAT) have proven to be versatile for a wide range of tasks by learning from both original features and graph structures.Here we present a graph attention model for predicting disease state from single-cell data on a large dataset of Multiple Sclerosis (MS) patients. MS is a disease of the central nervous system that can be difficult to diagnose. We train our model on single-cell data obtained from blood and cerebrospinal fluid (CSF) for a cohort of seven MS patients and six healthy adults (HA), resulting in 66,667 individual cells. We achieve 92 % accuracy in predicting MS, outperforming other state-of-the-art methods such as a graph convolutional network and a random forest classifier. Further, we use the learned graph attention model to get insight into the features (cell types and genes) that are important for this prediction. The graph attention model also allow us to infer a new feature space for the cells that emphasizes the differences between the two conditions. Finally we use the attention weights to learn a new low-dimensional embedding that can be visualized. To the best of our knowledge, this is the first effort to use graph attention, and deep learning in general, to predict disease state from single-cell data. We envision applying this method to single-cell data for other diseases. * Both authors contributed equally to this research.

show abstract

Section: Methodsmentioning

confidence: 99%

Disease state prediction from single-cell data using graph attention networks

Ravindra

Sehanobish

Pappalardo

et al. 2020

Proceedings of the ACM Conference on Health, Inference, and Learning

View full text Add to dashboard Cite

show abstract

“…What's more, they just sample nodes or edges from the whole graph to form the minibatches in each layer iteratively (Zeng et al, 2020), which causes "neighbor explosion" problem and leads to a large computation complexity. To solve this problem, some heuristic based methods which make subgraph sampling as a preprocessing step have been proposed (Chiang et al, 2019;Zeng et al, 2019). But the sampling strategies of these methods introduce the non-identical node sampling probability and bias.…”

Section: Spectral Methods Define the Convolution Operator Based Onmentioning

confidence: 99%

“…But it has limitations on scaling to large graphs and is difficult to be trained in a minibatch setting (Zeng et al, 2020). To address the problem of scaling GCN to large graphs, layer sampling methods (Chen et al, 2018a;Ying et al, 2018;Chen et al, 2018b;Gao et al, 2018;Huang et al, 2018) and subgraph sampling methods (Chiang et al, 2019;Zeng et al, 2019;Zeng et al, 2020) have been proposed. They are designed for efficient minibatch training for GCN (Zeng et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…They just sample nodes or edges from the whole graph to form the minibatches in each layer and then do the forward and backward propagation on the sampled graph. Subgraph sampling methods which perform the subgraph sampling before training the GCN reduce the problem of large computational cost in the node/edge sampling in each layer (Chiang et al, 2019;Zeng et al, 2019;Zeng et al, 2020). However, the filter size of above spectral methods is based on the size of the full graph or the size of the sampled subgraph which is not flexible enough for different sizes of input.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Star Topology Convolution for Graph Representation Learning

Wang¹,

Feng²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

<p>We present a novel graph convolutional method called star topology convolution (STC). This method makes graph convolution more similar to conventional convolutional neural networks (CNNs) in Euclidean feature space. Unlike most existing spectral methods, this method learns subgraphs which have a star topology rather than a fixed graph. Due to the good properties of a star topology, STC is graph/subgraph free. It has fewer parameters in its convolutional filter and is inductive so that it is more flexible and can be applied to large and evolving graphs. Similar to CNNs in Euclidean feature space, the convolutional filter is learnable and localized and maintains a good weight sharing property. To test the method, STC was compared to state-of-the-art spectral methods and spatial methods in a supervised learning setting on five benchmark datasets: Cora, Citeseer, Pubmed, Ogbn-Arxiv, and Ogbn-MAG. The experiment results show that STC outperforms other methods especially on large graphs. In an essential protein identification task, STC also outperforms state-of-the-art essential protein identification methods.<br></p>

show abstract

“…In this experiment, λ 1 is set to 0.004 and λ 2 is set to 0.05 according the grid search on λ 1 ∈ {0, 0.002, 0.004, 0.006, 0.008, 0.01} and λ 2 ∈ {0.01, 0.02, 0.03, 0.04, 0.05}. For all GNN models compared in this experiment, the updated news representation and [8] 97.3 ± 0.2 -FastGCN [24] -93.7 GeniePath [42] 98.5 -Cluster-GCN [43] 99.36 96.60 GaAN [25] 98.71 ± 0.02 96.83±0. 03 GAIN(ours) 99.37 ± 0.01 97.03±0.03 user representation are then concatenated, and this concatenated vector is fed through 3 fully connected layers to get the output.…”

Section: Experimental Set-upmentioning

confidence: 99%

GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning Over Large-Scale Graphs

Weng

Chen

Liang

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have led to state-of-the-art performance on a variety of machine learning tasks such as recommendation, node classification and link prediction. Graph neural network models generate node embeddings by merging nodes features with the aggregated neighboring nodes information. Most existing GNN models exploit a single type of aggregator (e.g., mean-pooling) to aggregate neighboring nodes information, and then add or concatenate the output of aggregator to the current representation vector of the center node. However, using only a single type of aggregator is difficult to capture the different aspects of neighboring information and the simple addition or concatenation update methods limit the expressive capability of GNNs. Not only that, existing supervised or semi-supervised GNN models are trained based on the loss function of the node label, which leads to the neglect of graph structure information. In this paper, we propose a novel graph neural network architecture, Graph Attention & Interaction Network (GAIN), for inductive learning on graphs. Unlike the previous GNN models that only utilize a single type of aggregation method, we use multiple types of aggregators to gather neighboring information in different aspects and integrate the outputs of these aggregators through the aggregator-level attention mechanism. Furthermore, we design a graph regularized loss to better capture the topological relationship of the nodes in the graph. Additionally, we first present the concept of graph feature interaction and propose a vector-wise explicit feature interaction mechanism to update the node embeddings. We conduct comprehensive experiments on two node-classification benchmarks and a real-world financial news dataset. The experiments demonstrate our GAIN model outperforms current state-of-the-art performances on all the tasks.

show abstract

Cluster-GCN

Cited by 705 publications

References 9 publications

Disease state prediction from single-cell data using graph attention networks

Disease state prediction from single-cell data using graph attention networks

Star Topology Convolution for Graph Representation Learning

GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning Over Large-Scale Graphs

Contact Info

Product

Resources

About