The Expressive Power of Graph Neural Networks as a Query Language

Abstract

In this paper we survey our recent results characterizing various graph neural network (GNN) architectures in terms of their ability to classify nodes over graphs, for classifiers based on unary logical formulas- or queries. We focus on the language FOC2, a well-studied fragment of FO. This choice is motivated by the fact that FOC2 is related to theWeisfeiler-Lehman (WL) test for checking graph isomorphism, which has the same ability as GNNs for distinguishing nodes on graphs. We unveil the exact relationship between FOC2 and GNNs in terms of node classification. To tackle this problem, we start by studying a popular basic class of GNNs, which we call AC-GNNs, in which the features of each node in a graph are updated, in successive layers, according only to the features of its neighbors. We prove that the unary FOC2 formulas that can be captured by an AC-GNN are exactly those that can be expressed in its guarded fragment, which in turn corresponds to graded modal logic. This result implies in particular that ACGNNs are too weak to capture all FOC2 formulas. We then seek for what needs to be added to AC-GNNs for capturing all FOC2. We show that it suffices to add readouts layers, which allow updating the node features not only in terms of its neighbors, but also in terms of a global attribute vector. We call GNNs with readouts ACR-GNNs. We also describe experiments that validate our findings by showing that, on synthetic data conforming to FOC2 but not to graded modal logic, AC-GNNs struggle to fit in while ACR-GNNs can generalise even to graphs of sizes not seen during training.

On the expressive power of message-passing neural networks as global feature map transformers

“…A notable exception is the work by Barceló et al [7,6], which inspired our present work. Barceló et al were the first to consider expressiveness of GNNs uniformly over all graphs (note, however, the earlier work of Hella et al [19] on similar message-passing distributed computation models).…”

Querying in the Age of Graph Databases and Knowledge Graphs

“…In the previous section, we considered this problem for regular expressions, but such patterns can be specified in other frameworks, ranging from logic-based declarative languages [17,45] to more procedural frameworks such as graph neural networks [58,72]. The goal of this part of the document is to show a recently established tight connection between these apparently different frameworks [18,60,84], which has interesting corollaries in terms of the use of declarative formalisms to specify patterns, versus the use of procedural formalisms to efficiently evaluate them.…”

“…The architecture for a graph neural network G defined in this section, that is referred to as an aggregate-combine graph neural network [18], turns G into a classifier [58,72]. But also G can be considered as a unary query that is true for a node 𝑢 of a vectorlabeled graph V if, and only if, the output of G is true for 𝑢, that is G(𝑢, V) = true.…”

“…Notice that FOC 2 is an extension of FO 2 as quantifier ∃𝑥 ≥𝑘 𝑥 𝜓 (𝑥) can be expressed in first-order logic but using 𝑘 variables. In [18], it is proved that there is a tight connection between a natural fragment of FOC 2 and aggregate-combine GNNs, since these two formalisms have the same expressive power as query languages. Such a characterization is based on a classical result showing that the Weisfeiler-Lehman test for graph isomorphism [83] has the same expressive power as FOC 2 [26], and a recently established result showing that the Weisfeiler-Lehman test bounds the expressiveness of aggregatecombine GNNs [60,84] .…”

“…Interestingly, the Weisfeiler-Lehman test can be formalized as a message-passing graph algorithm [49], which is an algorithmic model intimately related with graph neural networks. For a more detailed, and formal, description of the results outlined in this section see [18,40], where also many other interesting questions about the relationship between formal logic and GNNs are posed.…”