Coarse-to-Fine Attention Models for Document Summarization

Ling, Jeffrey; Rushton, Gérard

doi:10.18653/v1/w17-4505

Cited by 35 publications

(37 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HRED: the HieRarchical Encoder-Decoder (HRED) with hierarchical attention mechanism. This architecture has been proven effective in several NLP tasks including summarization (Ling and Rush 2017), headline generation (Tan, Wan, and Xiao 2017), and text generation (Li, Luong, and Jurafsky 2015). Here we keep the LSTMs size as 500 for fairness and set the number of the word encoder and sentence encoder layers as 1 and the decoder layer as 2.…”

Section: Baselines and Ablationsmentioning

confidence: 99%

“…We design our framework to explicitly tackle the above mentioned challenges by using a data-driven approach to learn to meet these requirements automatically. More specifically, we employ the hierarchical encoder-decoder network, which has already shown potentials to tackle long sequential input (Tan, Wan, and Xiao 2017;Ling and Rush 2017), as the base model for building our framework. On top of the hierarchical encoding structure, we propose the dynamic attention mechanism to combine sentence-level and word-level attentions varying at each recurrent time step to generate a more readable sequence.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generating Distractors for Reading Comprehension Questions from Real Examinations

Gao

King

et al. 2019

AAAI

View full text Add to dashboard Cite

We investigate the task of distractor generation for multiple choice reading comprehension questions from examinations. In contrast to all previous works, we do not aim at preparing words or short phrases distractors, instead, we endeavor to generate longer and semantic-rich distractors which are closer to distractors in real reading comprehension from examinations. Taking a reading comprehension article, a pair of question and its correct option as input, our goal is to generate several distractors which are somehow related to the answer, consistent with the semantic context of the question and have some trace in the article. We propose a hierarchical encoderdecoder framework with static and dynamic attention mechanisms to tackle this task. Specifically, the dynamic attention can combine sentence-level and word-level attention varying at each recurrent time step to generate a more readable sequence. The static attention is to modulate the dynamic attention not to focus on question irrelevant sentences or sentences which contribute to the correct option. Our proposed framework outperforms several strong baselines on the first prepared distractor generation dataset of real reading comprehension questions. For human evaluation, compared with those distractors generated by baselines, our generated distractors are more functional to confuse the annotators.

show abstract

Section: Baselines and Ablationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generating Distractors for Reading Comprehension Questions from Real Examinations

Gao

King

et al. 2019

AAAI

View full text Add to dashboard Cite

show abstract

“…Reinforce-select (RS) (Ling and Rush, 2017;Chen and Bansal, 2018) utilizes reinforcement learning to approximate the marginal likelihood. Specifically, it is trained to maximize a lower bound of the likelihood by applying the Jensen inequalily:…”

Section: Reinforce-selectmentioning

confidence: 99%

Select and Attend: Towards Controllable Content Selection in Text Generation

Shen¹,

Suzuki

Inui

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Many text generation tasks naturally contain two steps: content selection and surface realization. Current neural encoder-decoder models conflate both steps into a black-box architecture. As a result, the content to be described in the text cannot be explicitly controlled. This paper tackles this problem by decoupling content selection from the decoder. The decoupled content selection is human interpretable, whose value can be manually manipulated to control the content of generated text. The model can be trained end-to-end without human annotations by maximizing a lower bound of the marginal likelihood. We further propose an effective way to trade-off between performance and controllability with a single adjustable hyperparameter. In both data-to-text and headline generation tasks, our model achieves promising results, paving the way for controllable content selection in text generation.

show abstract

“…A short summary/headline (sequence of words) [9], [10], [14], [52] [12], [53]- [58] Question Generation A single word answer from a document or the start and end index of the answer in the document…”

Section: Machine Translationmentioning

confidence: 99%

Deep Reinforcement Learning for Sequence-to-Sequence Models

Keneshloo

Shi

Ramakrishnan

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

127

View full text Add to dashboard Cite

In recent times, sequence-to-sequence (seq2seq) models have gained a lot of popularity and provide state-ofthe-art performance in a wide variety of tasks such as machine translation, headline generation, text summarization, speech to text conversion, and image caption generation. The underlying framework for all these models is usually a deep neural network comprising an encoder and a decoder. Although simple encoderdecoder models produce competitive results, many researchers have proposed additional improvements over these seq2seq models, e.g., using an attention-based model over the input, pointergeneration models, and self-attention models. However, such seq2seq models suffer from two common problems: 1) exposure bias and 2) inconsistency between train/test measurement. Recently, a completely novel point of view has emerged in addressing these two problems in seq2seq models, leveraging methods from reinforcement learning (RL). In this survey, we consider seq2seq problems from the RL point of view and provide a formulation combining the power of RL methods in decision-making with seq2seq models that enable remembering long-term memories. We present some of the most recent frameworks that combine concepts from RL and deep neural networks. Our work aims to provide insights into some of the problems that inherently arise with current approaches and how we can address them with better RL models. We also provide the source code for implementing most of the RL models discussed in this paper to support the complex task of abstractive text summarization and provide some targeted experiments for these RL models, both in terms of performance and training time.

show abstract

Coarse-to-Fine Attention Models for Document Summarization

Cited by 35 publications

References 11 publications

Generating Distractors for Reading Comprehension Questions from Real Examinations

Generating Distractors for Reading Comprehension Questions from Real Examinations

Select and Attend: Towards Controllable Content Selection in Text Generation

Deep Reinforcement Learning for Sequence-to-Sequence Models

Contact Info

Product

Resources

About