Beyond Low-frequency Information in Graph Convolutional Networks

Abstract

Graph neural networks (GNNs) have been proven to be effective in various network-related tasks. Most existing GNNs usually exploit the low-frequency signals of node features, which gives rise to one fundamental question: is the low-frequency information all we need in the real world applications? In this paper, we first present an experimental investigation assessing the roles of low-frequency and high-frequency signals, where the results clearly show that exploring low-frequency signal only is distant from learning an effective node representation in different scenarios. How can we adaptively learn more information beyond low-frequency information in GNNs? A well-informed answer can help GNNs enhance the adaptability. We tackle this challenge and propose a novel Frequency Adaptation Graph Convolutional Networks (FAGCN) with a self-gating mechanism, which can adaptively integrate different signals in the process of message passing. For a deeper understanding, we theoretically analyze the roles of low-frequency signals and high-frequency signals on learning node representations, which further explains why FAGCN can perform well on different types of networks. Extensive experiments on six real-world networks validate that FAGCN not only alleviates the over-smoothing problem, but also has advantages over the state-of-the-arts.

Signed Graph Neural Networks: A Frequency Perspective

“…Popular spectral domain designs of graph neural networks including ChebNet (Defferrard et al, 2016), GCN (Kipf & Welling, 2017) and their further improvements such as AGCN (Li et al, 2018), Simplified GCN (Wu et al, 2019), and FAGCN (Bo et al, 2021) are based on the spectral analysis of signals (features) defined on an unsigned graph. The spectral analysis of graph signals has been studied under the umbrella of the graph signal processing (GSP) framework (Shuman et al, 2013;Ortega et al, 2018).…”

“…The former behaves like a low-pass filtering and can be viewed as a signed graph counterpart of the vanilla GCN (Kipf & Welling, 2017). The latter is able to retain high-frequency information and can be viewed as a signed graph counterpart of FAGCN (Bo et al, 2021).…”

“…As has been noted in FAGCN (Bo et al, 2021), besides low frequency components, it is beneficial to incorporate the high frequency components during feature aggregation as well. Based on this idea and the frequency interpretation on signed graphs, we extend FAGCN to signed graphs by considering low as well as high frequency information.…”

“…Since the features are not given for the nodes, we use truncated SVD of the symmetric adjacency matrix with dimension of 64 as input features. For comparison, we use traditional unsigned GNN designs namely GCN (Kipf & Welling, 2017), GraphSAGE (Hamilton et al, 2017), and FAGCN (Bo et al, 2021). For these methods, we do not consider the sign of the links, since the signed edge information is not applicable for these methods.…”

“…Recall that the spectral domain analysis of unsigned graphs has been widely used to develop GNN architectures. Many well-known GNNs including spectral-GNN (Bruna et al, 2014), ChebNet (Defferrard et al, 2016), GCN (Kipf & Welling, 2017), AGCN (Li et al, 2018), and FAGCN (Bo et al, 2021) rely on spectral domain analysis. These designs Preprint.…”