Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

Abstract

The celebrated Sequence to Sequence learning (Seq2Seq) technique and its numerous variants achieve excellent performance on many tasks. However, many machine learning tasks have inputs naturally represented as graphs; existing Seq2Seq models face a significant challenge in achieving accurate conversion from graph form to the appropriate sequence. To address this challenge, we introduce a novel general end-to-end graph-to-sequence neural encoder-decoder model that maps an input graph to a sequence of vectors and uses an attention-based LSTM method to decode the target sequence from these vectors. Our method first generates the node and graph embeddings using an improved graph-based neural network with a novel aggregation strategy to incorporate edge direction information in the node embeddings. We further introduce an attention mechanism that aligns node embeddings and the decoding sequence to better cope with large graphs. Experimental results on bAbI, Shortest Path, and Natural Language Generation tasks demonstrate that our model achieves state-of-the-art performance and significantly outperforms existing graph neural networks, Seq2Seq, and Tree2Seq models; using the proposed bi-directional node embedding aggregation strategy, the model can converge rapidly to the optimal performance.

Global Context Enhanced Graph Neural Networks for Session-based Recommendation

“…GNN-based Methods. Recent years have seen a surge of interests in Graph Neural Networks [2,3,6,9,30] and as a result various GNN methods have been utilized for improving recommendation system. Several GNN-based models have been proposed to learn item representations for the session-based recommendation [1,15,[25][26][27]29].…”

GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search

“…Graph Neural Networks. Recently, GNNs (Hamilton, Ying, and Leskovec 2017;Li et al 2015;Kipf and Welling 2016;Xu et al 2018) have become a hot research topic since their strengths in learning structure data. Various applications in different domains such as chemistry biology (Duvenaud et al 2015), computer vision (Norcliffe-Brown, Vafeias, andParisot 2018), natural language processing (Xu et al 2018;Chen, Wu, and Zaki 2019b) have demonstrated the effectiveness of GNNs.…”

“…Recently, GNNs (Hamilton, Ying, and Leskovec 2017;Li et al 2015;Kipf and Welling 2016;Xu et al 2018) have become a hot research topic since their strengths in learning structure data. Various applications in different domains such as chemistry biology (Duvenaud et al 2015), computer vision (Norcliffe-Brown, Vafeias, andParisot 2018), natural language processing (Xu et al 2018;Chen, Wu, and Zaki 2019b) have demonstrated the effectiveness of GNNs. In program scenario, compared with the early works to represent programs with abstract syntax tree (Alon et al 2018(Alon et al , 2019Liu et al 2020b), more works have already attempted to use graphs (Allamanis, Brockschmidt, and Khademi 2017) to learn the semantics for various applications, e.g., source code summarization (Liu et al 2020a;Fernandes, Allamanis, and Brockschmidt 2018), vulnerability detection (Zhou et al 2019), type inference (Allamanis et al 2020).…”

RDF-to-Text Generation with Graph-augmented Structural Neural Encoders

“…Song et al [50] explored the effectiveness of AMR as a semantic representation for neural machine translation based on a graph recurrent network. Previous studies [12,21,60] applied a bidirectional graph encoder to encode an input graph and then used an attention-based LSTM [27] decoder. To capture longer-range dependencies, Song et al [51] employed an LSTM to process the state transitions of the graph encoder outputs.…”