GPT-too: A Language-Model-First Approach for AMR-to-Text Generation

Abstract

Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs. Existing approaches to generating text from AMR have focused on training sequenceto-sequence or graph-to-sequence models on AMR annotated data only. In this paper, we propose an alternative approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques on the English LDC2017T10 dataset, including the recent use of transformer architectures. In addition to the standard evaluation metrics, we provide human evaluation experiments that further substantiate the strength of our approach.

Logic2Text: High-Fidelity Natural Language Generation from Logical Forms

“…The Transformer+copy model obtains better performance than the Graph2seq+copy model, as the Transformer architecture is indeed a graph neural network with self-attention as aggregation function over the neighbors and regards the input as a fully-connected graph. Recent works (Lin et al, 2019;Rogers et al, 2020;Mager et al, 2020) have shown that Transformer-based structure can capture hierarchical syntactic structures and graph representations. The GPT-2 model obtains the best performance among all with a significantly larger improvement.…”

A Human Evaluation of AMR-to-English Generation Systems

“…Ribeiro et al (2019) conducted a Mechanical Turk evaluation to compare their best graph encoder model with a sequence-to-sequence baseline, finding that their model performs better on both meaning similarity between the generated sentence and the gold reference, and readability of the generated sentence. Mager et al (2020) carry out a human evaluation of overall quality, comparing their GPT-2-based system to three others (Guo et al, 2019;Ribeiro et al, 2019;Zhu et al, 2019), all of which are also evaluated in our experiment. For the three systems included in both our evaluation and theirs, the relative results are comparable; Mager et al (2020) find their own system to be better than all three.…”

“…Mager et al (2020) carry out a human evaluation of overall quality, comparing their GPT-2-based system to three others (Guo et al, 2019;Ribeiro et al, 2019;Zhu et al, 2019), all of which are also evaluated in our experiment. For the three systems included in both our evaluation and theirs, the relative results are comparable; Mager et al (2020) find their own system to be better than all three.…”

Online Back-Parsing for AMR-to-Text Generation

“…These approaches show comparable or better results than early neural models (Konstas et al, 2017). However, recent neural approaches (Song et al, 2018;Zhu et al, 2019;Cai and Lam, 2020;Wang et al, 2020;Mager et al, 2020) have demonstrated the state-of-the-art performances thanks to the use of contextualized embeddings.…”