Topic-to-Essay Generation with Neural Networks

Abstract

We focus on essay generation, which is a challenging task that generates a paragraph-level text with multiple topics.Progress towards understanding different topics and expressing diversity in this task requires more powerful generators and richer training and evaluation resources. To address this, we develop a multi-topic aware long short-term memory (MTA-LSTM) network.In this model, we maintain a novel multi-topic coverage vector, which learns the weight of each topic and is sequentially updated during the decoding process.Afterwards this vector is fed to an attention model to guide the generator.Moreover, we automatically construct two paragraph-level Chinese essay corpora, 305,000 essay paragraphs and 55,000 question-and-answer pairs.Empirical results show that our approach obtains much better BLEU score compared to various baselines.Furthermore, human judgment shows that MTA-LSTM has the ability to generate essays that are not only coherent but also closely related to the input topics.

Generative Adversarial Networks and Its Applications in Biomedical Informatics

“…In the perspective of the probability and statistical theory, a discriminant model is a method of modeling the relationship between unknown data y and known data x. A generating model refers to a model that can randomly generate observations, especially under the condition of given some implicit parameters (10). Due to the invention of algorithms such as Back Propagation (BP) and Dropout, the discriminant model has been evolved rapidly.…”

Enhancing Topic-to-Essay Generation with External Commonsense Knowledge

“…The automatic evaluation of TEG remains an open and tricky question since the output is highly flexible. Previous work (Feng et al, 2018) only adopts BLEU (Papineni et al, 2002) score based on ngram overlap to perform evaluation. However, it is unreasonable to only use BLEU for evaluation because TEG is an extremely flexible task.…”

“…Automatic topic-to-essay generation (TEG) aims to compose novel, diverse, and topic-consistent paragraph-level text for several given topics. Feng et al (2018) are the first to propose the TEG task and they utilize coverage vector to integrate topic information. However, the performance is unsatisfactory, showing that more effective model architecture needs to be explored, which is also the original intention of our work.…”

“…Despite its wide applications described above, the progress in the TEG task lags behind other generation tasks such as machine translation (Bahdanau et al, 2014) or text summarization (Rush et al, 2015). Feng et al (2018) are the first to propose the TEG task and they utilize coverage vector * Equal Contribution. Figure 1: Toy illustration of the information volume on three different text generation tasks, which shows that the source information is extremely insufficient compared to the target output on the TEG task.…”

A Hierarchical Attention Based Seq2Seq Model for Chinese Lyrics Generation

“…In this paper, lyrics generation is modeled as a specific text-to-text generation with previous sentences as input. Similar tasks including Chinese poetry generation (Wang et al, 2016), essay generation (Feng et al, 2018) and comment generation have been extensively studied. Chinese poetry generation generate a kind of hierarchical text with strict format which often has a fixed number of sentences and each sentence has a fixed number of words.…”