Weisfeiler and Lehman Go Cellular: CW Networks

Abstract

Graph Neural Networks (GNNs) are limited in their expressive power, struggle with long-range interactions and lack a principled way to model higher-order structures. These problems can be attributed to the strong coupling between the computational graph and the input graph structure. The recently proposed Message Passing Simplicial Networks naturally decouple these elements by performing message passing on the clique complex of the graph. Nevertheless, these models are severely constrained by the rigid combinatorial structure of Simplicial Complexes (SCs). In this work, we extend recent theoretical results on SCs to regular Cell Complexes, topological objects that flexibly subsume SCs and graphs. We show that this generalisation provides a powerful set of graph "lifting" transformations, each leading to a unique hierarchical message passing procedure. The resulting methods, which we collectively call CW Networks (CWNs), are strictly more powerful than the WL test and, in certain cases, not less powerful than the 3-WL test. In particular, we demonstrate the effectiveness of one such scheme, based on rings, when applied to molecular graph problems. The proposed architecture benefits from provably larger expressivity than commonly used GNNs, principled modelling of higherorder signals and from compressing the distances between nodes. We demonstrate that our model achieves state-of-the-art results on a variety of molecular datasets.

Graph Neural Networks with Learnable Structural and Positional Representations

“…Finally, LSPE enables PNA to achieve comparable performance to SOTA on MOLPCBA while boosting its performance when no PE was used. We note here that ZINC scores can even be boosted beyond LSPE's SOTA when expert prior knowledge is used (Bouritsas et al, 2020;Bodnar et al, 2021) while Graphormer (Ying et al, 2021) achieved the top score on MOLPCBA when pre-trained on a very large (3.8M graphs) molecular dataset. To ensure fair comparison with other scores, we did not use these two results in Table 2.…”

“…You et al (2019) proposed learnable position-aware embeddings based on random anchor sets of nodes, where the random selection of anchors has its limitations, which makes their approach less generalizable on inductive tasks. There also exists methods that encode prior information about a class of graphs of interest such as rings for molecules (Bouritsas et al, 2020;Bodnar et al, 2021) which make MP-GNNs more expressive. But the prior information regarding graph sub-structures needs to be pre-computed, and sub-graph matching and counting require O(n k ) for k-tuple sub-structure.…”

“…However, the random selection of anchors has its limitations, which makes their approach less generalizable on inductive tasks. Bouritsas et al (2020); Bodnar et al (2021) introduced hybrid GNNs based on the WL-test and the message-passing aggregation mechanism. These networks use prior knowledge about a class of graphs of interest such as rings for molecules and cliques for social networks.…”

“…Another approach is to compute higher-order node-tuple aggregations such as in WL-based GNNs (Maron et al, 2019;Chen et al, 2019); though these models are computationally more expensive to scale than MP-GNNs, even for medium-sized graphs (Dwivedi et al, 2020). An alternative technique is to consider a global positioning of the nodes in the graph that can encode a graph-based distance between the nodes (You et al, 2019;Dwivedi et al, 2020;Li et al, 2020b;Dwivedi & Bresson, 2021), or can inform about specific sub-structures (Bouritsas et al, 2020;Bodnar et al, 2021).…”

Subgraph Permutation Equivariant Networks

“…Bodnar et al (2021b) introduce a WL test on simplicial complexes and incorporate this into a message passing scheme. Bodnar et al (2021a) extend work on simplicial complexes to cell complexes, which subsume simplicial complexes.…”