Transfer Learning for Deep Learning on Graph-Structured Data

Abstract

Graphs provide a powerful means for representing complex interactions between entities. Recently, new deep learning approaches have emerged for representing and modeling graph-structured data while the conventional deep learning methods, such as convolutional neural networks and recurrent neural networks, have mainly focused on the grid-structured inputs of image and audio. Leveraged by representation learning capabilities, deep learning-based techniques can detect structural characteristics of graphs, giving promising results for graph applications. In this paper, we attempt to advance deep learning for graph-structured data by incorporating another component: transfer learning. By transferring the intrinsic geometric information learned in the source domain, our approach can construct a model for a new but related task in the target domain without collecting new data and without training a new model from scratch. We thoroughly tested our approach with large-scale real-world text data and confirmed the effectiveness of the proposed transfer learning framework for deep learning on graphs. According to our experiments, transfer learning is most effective when the source and target domains bear a high level of structural similarity in their graph representations.

A Review of Graph Neural Networks and Their Applications in Power Systems

“…The proposed framework views towards the important graph information and activates the capturing of it as the goals of transferable GNNs training, which motivates the design of frameworks of GNNs. In [64], a model containing transfer learning and GNNs is proposed to solve the related tasks in the target domain without training a new model from scratch by transferring the natural geometric information learned in the source domain.…”

Leveraging universality of jet taggers through transfer learning

“…The two main avenues to achieve this goal is either through the retraining of a deep neural network while freezing the weights of its initial layers [79], or through the fine-tuning of the model [80][81][82]. While most existing applications are based on convolutional or recurrent neural networks, the development of deep learning on graph structured data has also seen advances in transfer learning applied on graph neural networks [83,84].…”

Transfer Graph Neural Networks for Pandemic Forecasting

“…More recently, this approach was utilized in the context of COVID-19 (Lampos et al 2020). In the context of graph representation learning, transfer learning has only been used to the best of our knowledge for classifying textual documents represented as graphs (Lee et al 2017), for traffic prediction (Mallick et al 2020), for semi-supervised classification (Yao et al 2020) and for designing GNNs that are robust to adversarial attacks (Tang et al 2020).…”