Exponentially Improving the Complexity of Simulating the Weisfeiler-Lehman Test with Graph Neural Networks

Aamand, Anders; Chen, Justin Y.; Indyk, Piotr; Narayanan, Shyam; Rubinfeld, Ronitt; Schiefer, Nicholas; Silwal, Sandeep; Wagner, Tal

doi:10.48550/arxiv.2211.03232

Cited by 1 publication

(12 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Those of [24] are polynomial. Both have recently been improved by [1], mainly showing that the messages only need to contain logarithmically many bits.…”

Section: Related Workmentioning

confidence: 99%

“…These are precisely the numbers that have a presentation as finite precision binary floating point numbers. We denote the set of dyadic rationals by Z 1 2 . We denote the binary representation of n ∈ N by bin(n).…”

Section: Preliminariesmentioning

confidence: 99%

“…L is rational if all its parameters a i , b i , t i in the minimal representation are dyadic rationals. 1 If L is rational, then its bitsize of bsize(L) is the sum of the bitsizes of all the parameters a i , b i , t i of the minimal representation. Oberserve that if L is continuous then it is Lipschitz continuous with Lipschitz constant max 0≤i≤n a i .…”

Section: Functions and Approximationsmentioning

confidence: 99%

“…where the formula seq and the terms entry, seqlen are from Lemma 3.12 and the term flog is from Lemma 3.13. Then for all q ∈ N we have ⟦ϕ⟧ a (m, q) = 1 ⇐⇒ q = (p (1) , . .…”

Section: And (2) Follows From (1) ⌟ Claimmentioning

confidence: 99%

“…Those of [24] are polynomial. Both have recently been improved by [1], mainly showing that the messages only need to contain logarithmically many bits.We are not the first to study the logical expressiveness of GNNs (see [12] for a recent survey). It was proved in [3] that all unary queries definable in the guarded fragment GC of the extension C of first-order logic by counting quantifiers ∃ ≥n x ("there exist at least n vertices x satisfying some formula") are computable by a GNN.…”

mentioning

confidence: 99%

See 4 more Smart Citations

The Descriptive Complexity of Graph Neural Networks

Grohe¹

2023

Preprint

View full text Add to dashboard Cite

We analyse the power of graph neural networks (GNNs) in terms of Boolean circuit complexity and descriptive complexity.We prove that the graph queries that can be computed by a polynomial-size bounded-depth family of GNNs are exactly those definable in the guarded fragment GFO+C of first-order logic with counting and with built-in relations. This puts GNNs in the circuit complexity class TC 0 . Remarkably, the GNN families may use arbitrary real weights and a wide class of activation functions that includes the standard ReLU, logistic "sigmoid", and hyperbolic tangent functions. If the GNNs are allowed to use random initialisation and global readout (both standard features of GNNs widely used in practice), they can compute exactly the same queries as bounded depth Boolean circuits with threshold gates, that is, exactly the queries in TC 0 .Moreover, we show that queries computable by a single GNN with piecewise linear activations and rational weights are definable in GFO+C without built-in relations. Therefore, they are contained in uniform TC 0 .(2) Q is computable in TC 0 .We mention that, following [2], we allow GNNs with random initialisation to also use a feature known as global readout, which means that in each message-passing round of a GNN computation, the vertices not only receive messages from their neighbours, but the aggregated state of all vertices. There is also a version of Theorem 1.1 for GNNs with global readout. Related WorkA fundamental result on the expressiveness of GNNs [24,32] states that two graphs are distinguishable by a GNN if and only if they are distinguishable by the 1-dimensional Weisfeiler-Leman (WL) algorithm, a simple combinatorial algorithm originally introduced as a graph isomorphism heuristics [23,31]. This result has had considerable impact on the subsequent development of GNNs, because it provides a yardstick for the expressiveness of GNN extensions (see [25]). Its generalisation to higher-order GNNs and higher-dimensional WL algorithms [24] even gives a hierarchy of increasingly more expressive formalisms against which such extensions can be compared. However, these results relating GNNs and their extensions to the WL algorithm only consider a restricted form of expressiveness, the power to distinguish two graphs. Furthermore, the results are non-uniform, that is, the distinguishing GNNs depend on the input graphs or at least on their size, and the GNNs may be arbitrarily large and deep. Indeed, the GNNs from the construction in [32] may be exponentially large in the graphs they distinguish. Those of [24] are polynomial. Both have recently been improved by [1], mainly showing that the messages only need to contain logarithmically many bits.We are not the first to study the logical expressiveness of GNNs (see [12] for a recent survey). It was proved in [3] that all unary queries definable in the guarded fragment GC of the extension C of first-order logic by counting quantifiers ∃ ≥n x ("there exist at least n vertices x satisfying some formula") are computable by a GNN. The ...

show abstract

“…Those of [24] are polynomial. Both have recently been improved by [1], mainly showing that the messages only need to contain logarithmically many bits.…”

Section: Related Workmentioning

confidence: 99%

Section: Preliminariesmentioning

confidence: 99%

Section: Functions and Approximationsmentioning

confidence: 99%

“…where the formula seq and the terms entry, seqlen are from Lemma 3.12 and the term flog is from Lemma 3.13. Then for all q ∈ N we have ⟦ϕ⟧ a (m, q) = 1 ⇐⇒ q = (p (1) , . .…”

Section: And (2) Follows From (1) ⌟ Claimmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations