Improving sentence compression by learning to predict gaze

Klerke, Sigrid; Goldberg, Yoav; Søgaard, Anders

doi:10.18653/v1/n16-1179

Cited by 84 publications

(104 citation statements)

References 20 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…Bollmann and Søgaard (2016) also use bi-LSTMs to frame spelling normalization as a characterbased sequence labelling task, performing character alignment as a preprocessing step. Multi-task learning was shown to be effective for a variety of NLP tasks, such as POS tagging, chunking, named entity recognition (Collobert et al, 2011) or sentence compression (Klerke et al, 2016). It has also been used in encoderdecoder architectures, typically for machine translation (Dong et al, 2015;Luong et al, 2016), though so far not with attentional decoders.…”

Section: Related Workmentioning

confidence: 99%

Learning attention for historical text normalization by learning to pronounce

Bollmann

Bingel

Søgaard

2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

Self Cite

View full text Add to dashboard Cite

Automated processing of historical texts often relies on pre-normalization to modern word forms. Training encoder-decoder architectures to solve such problems typically requires a lot of training data, which is not available for the named task. We address this problem by using several novel encoder-decoder architectures, including a multi-task learning (MTL) architecture using a grapheme-to-phoneme dictionary as auxiliary data, pushing the state-of-theart by an absolute 2% increase in performance. We analyze the induced models across 44 different texts from Early New High German. Interestingly, we observe that, as previously conjectured, multi-task learning can learn to focus attention during decoding, in ways remarkably similar to recently proposed attention mechanisms. This, we believe, is an important step toward understanding how MTL works.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning attention for historical text normalization by learning to pronounce

Bollmann

Bingel

Søgaard

2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

Self Cite

View full text Add to dashboard Cite

show abstract

“…As reported in recent papers (Klerke et al, 2016;Wang et al, 2017), the F 1 scores of Tagger match or exceed those of the Seq2Seq-based methods. The compressed sentence of the first example in Table 3 output by Tagger is ungrammatical.…”

Section: Discussionmentioning

confidence: 55%

Higher-Order Syntactic Attention Network for Longer Sentence Compression

Kamigaito¹,

Hayashi²,

Hirao³

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

Sentence compression methods based on LSTM can generate fluent compressed sentences. However, the performance of these methods is significantly degraded when compressing long sentences since it does not explicitly handle syntactic features. To solve this problem, we propose a higher-order syntactic attention network (HiSAN) that can handle higher-order dependency features as an attention distribution on LSTM hidden states. Furthermore, to avoid the influence of incorrect parse results, we train HiSAN by maximizing the probability of a correct output together with the attention distribution. Experiments on the Google sentence compression dataset show that our method achieved the best performance in terms of F 1 as well as ROUGE-1,2 and L scores, 83.2, 82.9, 75.8 and 82.7, respectively. In subjective evaluations, HiSAN outperformed baseline methods in both readability and informativeness.

show abstract

“…Our second baseline is a cascading, three-layered LSTM, as described by (Klerke et al, 2016). See §3 for hyper-parameters.…”

Section: Evaluation Metricsmentioning

confidence: 99%

Sluice Resolution without Hand-Crafted Features over Brittle Syntax Trees

Rønning

Hardt

Søgaard

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

Self Cite

View full text Add to dashboard Cite

Sluice resolution in English is the problem of finding antecedents of wh-fronted ellipses. Previous work has relied on handcrafted features over syntax trees that scale poorly to other languages and domains; in particular, to dialogue, which is one of the most interesting applications of sluice resolution. Syntactic information is arguably important for sluice resolution, but we show that multi-task learning with partial parsing as auxiliary tasks effectively closes the gap and buys us an additional 9% error reduction over previous work. Since we are not directly relying on features from partial parsers, our system is more robust to domain shifts, giving a 26% error reduction on embedded sluices in dialogue.

show abstract

Improving sentence compression by learning to predict gaze

Abstract: We show how eye-tracking corpora can be used to improve sentence compression models, presenting a novel multi-task learning algorithm based on multi-layer LSTMs. We obtain performance competitive with or better than state-of-the-art approaches.

Cited by 84 publications

References 20 publications

Learning attention for historical text normalization by learning to pronounce

Learning attention for historical text normalization by learning to pronounce

Higher-Order Syntactic Attention Network for Longer Sentence Compression

Sluice Resolution without Hand-Crafted Features over Brittle Syntax Trees

Contact Info

Product

Resources

About