Investigating and Mitigating Degree-Related Biases in Graph Convoltuional Networks

Abstract

Graph Convolutional Networks (GCNs) show promising results for semi-supervised learning tasks on graphs, thus become favorable comparing with other approaches. Despite the remarkable success of GCNs, it is difficult to train GCNs with insufficient supervision. When labeled data are limited, the performance of GCNs becomes unsatisfying for low-degree nodes. While some prior work analyze successes and failures of GCNs on the entire model level, profiling GCNs on individual node level is still underexplored. In this paper, we analyze GCNs in regard to the node degree distribution. From empirical observation to theoretical proof, we confirm that GCNs are biased towards nodes with larger degrees with higher accuracy on them, even if high-degree nodes are underrepresented in most graphs. We further develop a novel Self-Supervised-Learning Degree-Specific GCN (SL-DSGCN) that mitigate the degree-related biases of GCNs from model and data aspects. Firstly, we propose a degree-specific GCN layer that captures both discrepancies and similarities of nodes with different degrees, which reduces the inner model-aspect biases of GCNs caused by sharing the same parameters with all nodes. Secondly, we design a self-supervised-learning algorithm that creates pseudo labels with uncertainty scores on unlabeled nodes with a Bayesian neural network. Pseudo labels increase the chance of connecting to labeled neighbors for low-degree nodes, thus reducing the biases of GCNs from the data perspective. Uncertainty scores are further exploited to weight pseudo labels dynamically in the stochastic gradient descent for SL-DSGCN. Experiments on three benchmark datasets show SL-DSGCN not only outperforms state-of-the-art selftraining/self-supervised-learning GCN methods, but also improves GCN accuracy dramatically for low-degree nodes.

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

“…We hope that new models will more regularly include datasets such as Hetionet during their development phase. More generally, and taking cues from the field of GNNs [25,26,44], new methods could be be developed which consider how best to learn meaningful representations for low-degree entities.…”

“…The issue of non-uniform graph connectivity (typically in homogenous graphs) has begun to be studied in parallel by the field of Graph Neural Networks (GNN), where researchers have shown that models learn low-quality representations, thus making more incorrect predictions, for low-degree vertices [26,25,44]. This has also been explored in the context of homogenous graph representation learning [3] and for random walks [23,36].…”

Friend Story Ranking with Edge-Contextual Local Graph Convolutions

“…Graph representation learning. Graph representation learning (GRL) aims to learn representations suitable for graph-based tasks, mainly including node/graph classification [21,48,50,62], and link prediction [17]. Early methods focused on non-attributed graphs, leveraging insights from language modeling [39] to learn embeddings which preserve node co-occurrence statistics on random walks [40].…”

Automated Self-Supervised Learning for Graphs

“…Like other machine learning models, GNNs can suffer from typical unfairness issues which may arise due to sensitive attributes, label parity issues, and more [55]. Moreover, GNNs can also suffer from degree-related biases [56]. Self-supervised learning (SSL) is often used to learn high-quality representations without supervision from labeled data sources, and is especially useful in low-resource settings or in pre-training/fine-tuning scenarios.…”