Unifying Graph Contrastive Learning with Flexible Contextual Scopes

Abstract

Graph contrastive learning (GCL) has recently emerged as an effective learning paradigm to alleviate the reliance on labelling information for graph representation learning. The core of GCL is to maximise the mutual information between the representation of a node and its contextual representation (i.e., the corresponding instance with similar semantic information) summarised from the contextual scope (e.g., the whole graph or 1hop neighbourhood). This scheme distils valuable self-supervision signals for GCL training. However, existing GCL methods still suffer from limitations, such as the incapacity or inconvenience in choosing a suitable contextual scope for different datasets and building biased contrastiveness. To address aforementioned problems, we present a simple self-supervised learning method termed Unifying Graph Contrastive Learning with Flexible Contextual Scopes (UGCL for short). Our algorithm builds flexible contextual representations with tunable contextual scopes by controlling the power of an adjacency matrix. Additionally, our method ensures contrastiveness is built within connected components to reduce the bias of contextual representations. Based on representations from both local and contextual scopes, UGCL optimises a very simple contrastive loss function for graph representation learning. Essentially, the architecture of UGCL can be considered as a general framework to unify existing GCL methods. We have conducted intensive experiments and achieved new state-of-theart performance in six out of eight benchmark datasets compared with self-supervised graph representation learning baselines. Our code has been open sourced 1 .

GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner

“…Baselines. We compare GraphMAE2 with state-of-the-art selfsupervised graph learning methods, including contrastive methods, GRACE [57], BGRL [39], CCA-SSG [54], and GGD [55] as well as a generative method GraphMAE [18]. Other methods are not compared because they are not scalable to large graphs, e.g., MVGRL [14], or the source code has not been released, e.g., In-foGCL [48].…”

“…Contrastive methods. Contrastive learning is an important way to learn representations in a self-supervised manner and has achieved successful practices in graph learning [14,29,35,37,42,51,55]. DGI [42] and InfoGraph [37] adopt the local-global mutual information maximization to learn node-level and graph-level representations.…”

“…GRACE [57], GraphCL [51], and GCA [58] learn the node or graph representation by maximizing agreement between different augmentations. GGD [55] analyzes the defect of existing contrastive methods (i.e., improper usage of Sigmoid function) and proposes a group discrimination paradigm.…”

Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation

“…f N is the initialized nodes representations and consists of entity embedding and level encoding. Previous works initialized representations randomly, which limits the effectiveness seriously [49]. Moreover, some additional nodes appear few times during the training and hard to find the best embeddings.…”

“…The improvement comes from two aspects. Firstly, it provides well pretrained parameters; Secondly, its tokenizer encodes each token (medical terminology) with a well pretrained embedding, which avoid Graph Transformers to propagate similar hidden states [49].…”