The WebNLG Challenge: Generating Text from RDF Data

Gardent, Claire; Shimorina, Anastasia; Narayan, Shashi; Perez-Beltrachini, Laura

doi:10.18653/v1/w17-3518

Cited by 259 publications

(327 citation statements)

References 15 publications

Supporting

Mentioning

285

Contrasting

Order By: Relevance

“…Marcheggiani and Perez Beltrachini (2018) show that explicitly encoding the structure of the graph is beneficial with respect to sequential encoding. They evaluate their model on two tasks, WebNLG (Gardent et al, 2017) and SR11Deep (Belz et al, 2011), but do not apply it to AMR benchmarks. Song et al (2018) and Beck et al (2018) apply recurrent neural networks to directly encode AMR graphs.…”

Section: Related Workmentioning

confidence: 99%

Enhancing AMR-to-Text Generation with Dual Graph Representations

Ribeiro¹,

Gardent

Gurevych³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

Generating text from graph-based data, such as Abstract Meaning Representation (AMR), is a challenging task due to the inherent difficulty in how to properly encode the structure of a graph with labeled edges. To address this difficulty, we propose a novel graph-to-sequence model that encodes different but complementary perspectives of the structural information contained in the AMR graph. The model learns parallel top-down and bottom-up representations of nodes capturing contrasting views of the graph. We also investigate the use of different node message passing strategies, employing different state-of-the-art graph encoders to compute node representations based on incoming and outgoing perspectives. In our experiments, we demonstrate that the dual graph representation leads to improvements in AMR-to-text generation, achieving state-ofthe-art results on two AMR datasets 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

Enhancing AMR-to-Text Generation with Dual Graph Representations

Ribeiro¹,

Gardent

Gurevych³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our impression is that recurring entity names in the training data coming from multiple reference texts for the same input lead to overfitting on the training vocabulary and to poor generalization to novel inputs. This is also reflected by the rather unsatisfying performance of subword-based approaches in the E2E 1 and WebNLG challenge (ADAPT system (Gardent et al, 2017b)).…”

Section: Input and Output Representationsmentioning

confidence: 99%

Sequence-to-Sequence Models for Data-to-Text Natural Language Generation: Word- vs. Character-based Processing and Output Diversity

Jagfeld¹,

Jenne²,

Vu³

2018

Proceedings of the 11th International Conference on Natural Language Generation

View full text Add to dashboard Cite

We present a comparison of word-based and character-based sequence-to-sequence models for data-to-text natural language generation, which generate natural language descriptions for structured inputs. On the datasets of two recent generation challenges, our models achieve comparable or better automatic evaluation results than the best challenge submissions. Subsequent detailed statistical and human analyses shed light on the differences between the two input representations and the diversity of the generated texts. In a controlled experiment with synthetic training data generated from templates, we demonstrate the ability of neural models to learn novel combinations of the templates and thereby generalize beyond the linguistic structures they were trained on.

show abstract

“…We evaluate the submitted systems by comparing them to a challenging baseline using automatic evaluation metrics (including novel text-based measures) as well as human evaluation (see Section 7). Note that, while there are other concurrent studies comparing a limited number of end-to-end NLG approaches (Novikova et al, 2017a Gardent et al, 2017a) which emerged during the E2E NLG Challenge, this is the first research to evaluate novel end-to-end generation at scale using human assessment.…”

Section: Introductionmentioning

confidence: 99%

Evaluating the state-of-the-art of End-to-End Natural Language Generation: The E2E NLG challenge

Dušek

Novikova

Rieser

2020

Computer Speech & Language

170

183

View full text Add to dashboard Cite

This paper provides a comprehensive analysis of the first shared task on End-to-End Natural Language Generation (NLG) and identifies avenues for future research based on the results. This shared task aimed to assess whether recent end-to-end NLG systems can generate more complex output by learning from datasets containing higher lexical richness, syntactic complexity and diverse discourse phenomena. Introducing novel automatic and human metrics, we compare 62 systems submitted by 17 institutions, covering a wide range of approaches, including machine learning architectures -with the majority implementing sequence-to-sequence models (seq2seq) -as well as systems based on grammatical rules and templates. Seq2seq-based systems have demonstrated a great potential for NLG in the challenge. We find that seq2seq systems generally score high in terms of word-overlap metrics and human evaluations of naturalness -with the winning Slug system (Juraska et al., 2018) being seq2seq-based.However, vanilla seq2seq models often fail to correctly express a given meaning representation if they lack a strong semantic control mechanism applied during decoding.Moreover, seq2seq models can be outperformed by hand-engineered systems in terms of overall quality, as well as complexity, length and diversity of outputs. This research has influenced, inspired and motivated a number of recent studies outwith the original competition, which we also summarise as part of this paper.

show abstract

The WebNLG Challenge: Generating Text from RDF Data

Cited by 259 publications

References 15 publications

Enhancing AMR-to-Text Generation with Dual Graph Representations

Enhancing AMR-to-Text Generation with Dual Graph Representations

Sequence-to-Sequence Models for Data-to-Text Natural Language Generation: Word- vs. Character-based Processing and Output Diversity

Evaluating the state-of-the-art of End-to-End Natural Language Generation: The E2E NLG challenge

Contact Info

Product

Resources

About