Data-to-Text Generation with Content Selection and Planning

Abstract

Recent advances in data-to-text generation have led to the use of large-scale datasets and neural network models which are trained end-to-end, without explicitly modeling what to say and in what order. In this work, we present a neural network architecture which incorporates content selection and planning without sacrificing end-to-end training. We decompose the generation task into two stages. Given a corpus of data records (paired with descriptive documents), we first generate a content plan highlighting which information should be mentioned and in which order and then generate the document while taking the content plan into account. Automatic and human-based evaluation experiments show that our model 1 outperforms strong baselines improving the state-of-the-art on the recently released ROTOWIRE dataset.

A Hierarchical Model for Data-to-Text Generation

“…To improve these models, a number of work [16,28,40] proposed innovating decoding modules based on planning and templates, to ensure factual and coherent mentions of records in generated descriptions. For example, Puduppully et al [28] propose a two-step decoder which first targets specific records and then use them as a plan for the actual text generation.…”

“…To improve these models, a number of work [16,28,40] proposed innovating decoding modules based on planning and templates, to ensure factual and coherent mentions of records in generated descriptions. For example, Puduppully et al [28] propose a two-step decoder which first targets specific records and then use them as a plan for the actual text generation. Similarly, Li et al [16] proposed a delayed copy mechanism where their decoder also acts in two steps: 1) using a classical LSTM decoder to generate delexicalized text and 2) using a pointer network [38] to replace placeholders by records from the input data.…”

“…Our contribution differs from previous work in several aspects. First, instead of flatly concatenating elements from the data-structure and encoding them as a sequence [18,28,39], we constrain the encoding to the underlying structure of the input data, so that the delimitation between entities remains clear throughout the process. Second, unlike all works in the domain, we exploit the Transformer architecture [36] and leverage its particularity to directly compare elements with each others in order to avoid arbitrary assumptions on their ordering.…”

“…The first one (module A in Figure 2) is called low-level encoder and encodes entities on the basis of their records; the second one (module B), called high-level encoder, encodes the data-structure on the basis of its underlying entities. In the low-level encoder, the traditional embedding layer is replaced by a record embedding layer as in [18,28,39]. We present in what follows the record embedding layer and introduce our two hierarchical modules.…”

“…• Puduppully-plan [28] acts in two steps: a first standard encoder-decoder generates a plan, i.e. a list of salient records from the table; a second standard encoder-decoder generates text from this plan.…”