An End-to-End Deep Learning Architecture for Graph Classification

Abstract

Neural networks are typically designed to deal with data in tensor forms. In this paper, we propose a novel neural network architecture accepting graphs of arbitrary structure. Given a dataset containing graphs in the form of (G,y) where G is a graph and y is its class, we aim to develop neural networks that read the graphs directly and learn a classification function. There are two main challenges: 1) how to extract useful features characterizing the rich information encoded in a graph for classification purpose, and 2) how to sequentially read a graph in a meaningful and consistent order. To address the first challenge, we design a localized graph convolution model and show its connection with two graph kernels. To address the second challenge, we design a novel SortPooling layer which sorts graph vertices in a consistent order so that traditional neural networks can be trained on the graphs. Experiments on benchmark graph classification datasets demonstrate that the proposed architecture achieves highly competitive performance with state-of-the-art graph kernels and other graph neural network methods. Moreover, the architecture allows end-to-end gradient-based training with original graphs, without the need to first transform graphs into vectors.

A graph neural network-based performance model for deep learning applications

“…We use the sum-pool operation, which involves adding the node representations. Following a technique similar to [22], we preserve the initial, intermediate node representations along with the final representations. The pooling operation involves summing the node representations at every convolution step, including the initial embeddings.…”

“…We arrived at this configuration after a parametric sweep of convolutional layers ranging from 0 to 8. Using the technique proposed in [22], we preserve the initial and intermediate embeddings for graph-level readout. To generate a representation for the entire graph, we sum across stages after every convolution and use a linear layer to obtain the runtime.…”

Distance metric learning for graph structured data

“…There are many other kernels including the shortest path (Borgwardt and Kriegel 2005)-, random walk (Vishwanathan et al 2010;Sugiyama and Borgwardt 2015;Zhang et al 2018b)-, and spectrum-based (Kondor and Borgwardt 2008;Kondor et al 2009;Kondor and Pan 2016;Verma and Zhang 2017) approaches. The Weisfeiler-Lehman (WL) kernel Shervashidze et al 2011), which is based on the graph isomorphism test, is a popular and empirically successful kernel that has been employed in many studies (Yanardag and Vishwanathan 2015;Niepert et al 2016;Narayanan et al 2017;Zhang et al 2018a). Again, all such approaches are unsupervised, and it is difficult to interpret results from the perspective of sub-structures of a graph.…”

“…The deep graph kernel (Yanardag and Vishwanathan 2015) incorporates neural language modeling, where decomposed sub-structures of a graph are regarded as sentences. The PATCHY-SAN (Niepert et al 2016) and DGCNN (Zhang et al 2018a) convert a graph to a tensor by using the WL-Kernel and convolute it. Several other studies also have combined popular convolution techniques with graph data (Tixier et al 2018;Atwood and Towsley 2016;Simonovsky and Komodakis 2017).…”

“…In this section, we compare the prediction accuracy of IGML with those of the Graphlet-Kernel (GK) ), Shortest-Path Kernel (SPK) (Borgwardt and Kriegel 2005), Random-Walk Kernel (RW) (Vishwanathan et al 2010), Weisfeiler-Lehman Kernel (WL) (Shervashidze et al 2011), and Deep Graph Convolutional Neural Network (DGCNN) (Zhang et al 2018a). We used the implementations available at the URLs in the footnote 1 .…”