Graph Transformer for Graph-to-Sequence Learning

Abstract

The dominant graph-to-sequence transduction models employ graph neural networks for graph representation learning, where the structural information is reflected by the receptive field of neurons. Unlike graph neural networks that restrict the information exchange between immediate neighborhood, we propose a new model, known as Graph Transformer, that uses explicit relation encoding and allows direct communication between two distant nodes. It provides a more efficient way for global graph structure modeling. Experiments on the applications of text generation from Abstract Meaning Representation (AMR) and syntax-based neural machine translation show the superiority of our proposed model. Specifically, our model achieves 27.4 BLEU on LDC2015E86 and 29.7 BLEU on LDC2017T10 for AMR-to-text generation, outperforming the state-of-the-art results by up to 2.2 points. On the syntax-based translation tasks, our model establishes new single-model state-of-the-art BLEU scores, 21.3 for English-to-German and 14.1 for English-to-Czech, improving over the existing best results, including ensembles, by over 1 BLEU.

Scene Graph Modification Based on Natural Language Commands

“…configurations of our model are superior to the baselines by a significant margin. Noticeably, DCGCN and graph transformer are strong baselines, delivering SOTA performance across tasks such as AMRto-text generation and syntax-based neural machine translation (Guo et al, 2019;Cai and Lam, 2019). We believe the larger number of edge types in our task impairs their capability.…”

“…In the "Modified GraphRNN" baseline (iii), we use the breadth-first-search (BFS) based node ordering to flatten the graph 4 , and use RNNs as the encoders (You et al, 2018) and a decoder similar to our systems. In the final two baselines, "Graph Transformer" (iv) and "Deep Convolutional Graph Networks" (DCGCN) (v), we use the Graph Transformers (Cai and Lam, 2019) and Deep Convolutional Graph Networks (Guo et al, 2019) to encode the source graph (the decoder is identical to ours).…”

“…Our work focuses on the update of scene graphs based on users' queries, while previous works model the modifications of semantic representations in multi-turn dialogue systems. Due to their effectiveness, GCNs and graph transformer have been used as graph encoder for graphto-sequence transduction in semantic-based text generation (Bastings et al, 2017;Beck et al, 2018;Guo et al, 2019;Cai and Lam, 2019;Song et al, 2018;Wu et al, 2020).…”

Online Back-Parsing for AMR-to-Text Generation

“…We take a graph Transformer model (Koncel-Kedziorski et al, 2019;Zhu et al, 2019;Cai and Lam, 2020;Wang et al, 2020) as our baseline. Previous work has proposed several variations of graph-Transformer.…”

“…In particular, graph neural networks (Beck et al, 2018;Song et al, 2018;Guo et al, 2019) and richer graph representations (Damonte and Cohen, 2019;Hajdik et al, 2019;Ribeiro et al, 2019) have been shown to give better performances than RNN-based models (Konstas et al, 2017) on linearized graphs. Subsequent work exploited graph Transformer (Zhu et al, 2019;Cai and Lam, 2020;Wang et al, 2020), achieving better performances by directly modeling the intercorrelations between distant node pairs with relation-aware global communication. Despite the progress on the encoder side, the current stateof-the-art models use a rather standard decoder: it functions as a language model, where each word is generated given only the previous words.…”