Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Abstract

There has been a surge of recent interest in learning representations for graph-structured data. Graph representation learning methods have generally fallen into three main categories, based on the availability of labeled data. The first, network embedding (such as shallow graph embedding or graph auto-encoders), focuses on learning unsupervised representations of relational structure. The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning. The third, graph neural networks, aims to learn differentiable functions over discrete topologies with arbitrary structure. However, despite the popularity of these areas there has been surprisingly little work on unifying the three paradigms. Here, we aim to bridge the gap between graph neural networks, network embedding and graph regularization models. We propose a comprehensive taxonomy of representation learning methods for graph-structured data, aiming to unify several disparate bodies of work. Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which generalizes popular algorithms for semi-supervised learning on graphs (e.g. GraphSage, Graph Convolutional Networks, Graph Attention Networks), and unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc) into a single consistent approach. To illustrate the generality of this approach, we fit over thirty existing methods into this framework. We believe that this unifying view both provides a solid foundation for understanding the intuition behind these methods, and enables future research in the area.

Learning to Model the Relationship Between Brain Structural and Functional Connectomes

“…GCNs are versatile signal and information processing architectures [15]- [17], which comprise stacked layers of graph (convolutional) filters followed by point-wise nonlinearities; see [17], [23], [26]- [29] for recent surveys and the references therein. From early spectral convolutions [15], [30] to distributed implementations of (equivalent) shift-invariant polynomial graph filters [19], [26], [31], GCNs integrate information from both the graph topology and nodal attributes to learn representations of network data.…”

“…From early spectral convolutions [15], [30] to distributed implementations of (equivalent) shift-invariant polynomial graph filters [19], [26], [31], GCNs integrate information from both the graph topology and nodal attributes to learn representations of network data. Indeed, the GRL paradigm is to learn low-dimensional embeddings of individual vertices, edges, or the graph itself [23], [32]- [34], which can then be used in e.g., (semi-supervised) node classification [15], link prediction [35], graph clustering [36], [37], and graph clas-sification [38]. Recently, GRL ideas have permeated to neuroimaging data analysis for behavioral state classification [39], to study the relationship between SC and FC [13], [40], and to extract representations for subject classification [41]- [43].…”

“…We first present the encompassing message passing formalism used for a broad class of GNNs [31]. As the GCN [15] is adopted for the encoder in the GRL pipeline, a brief review of graph convolutional models is provided as well; see also [17], [23], [27]- [29] for comprehensive surveys with further details.…”

“…Here we introduce the proposed graph encoder-decoder system used to model the relationship between structural and functional brain connectomes; see also the schematic description of the architecture in Figure 2 and the recent survey [23]. The top branch in Figure 2 implements a regression model to reconstruct FC from SC.…”

“…Our overall architecture can be viewed as a graph encoderdecoder system inspired by the graph autoencoder model [22]; see also [23]. For each RoI in the input SC graph, the encoder outputs low-dimensional node embeddings that integrate both nodal attributes (when available) and the local graph topology information.…”