SCAR: Sentence Compression using Autoencoders for Reconstruction

Malireddy, Chanakya; Maniar, Tirth; Shrivastava, Manish

doi:10.18653/v1/2020.acl-srw.13

Cited by 7 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Self-consistency Learning has been adopted in the field of Natural Language Processing in recent years for tasks like text encoding (Li et al, 2015), sentence compression and summarization (Baziotis et al, 2019;Malireddy et al, 2020), NER (Iovine et al, 2022), and data-to-text generation . Models that leverage self-consistency learning usually include two modules that are reversals of each other.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model

Wang,

Zhu

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Question generation is a widely used data augmentation approach with extensive applications, and extracting qualified candidate answers from context passages is a critical step for most question generation systems. However, existing methods for candidate answer extraction are reliant on linguistic rules or annotated data that face the partial annotation issue and challenges in generalization. To overcome these limitations, we propose a novel unsupervised candidate answer extraction approach that leverages the inherent structure of context passages through a Differentiable Masker-Reconstructor (DMR) Model with the enforcement of self-consistency for picking up salient information tokens. We curated two datasets with exhaustively-annotated answers and benchmark a comprehensive set of supervised and unsupervised candidate answer extraction methods. We demonstrate the effectiveness of the DMR model by showing its performance is superior among unsupervised methods and comparable to supervised methods. Our code and data are publicly available at https://edillower.github.io/.

show abstract

Section: Related Workmentioning

confidence: 99%

Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model

Wang,

Zhu

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…Another commonly proposed unsupervised framework is to use autoencoders and reconstruction objectives (Miao and Blunsom, 2016;Févry and Phang, 2018;Malireddy et al, 2020). These approaches are based on the assumption that a good sentence compression is one from which the original sentence can be inferred.…”

Section: Unsupervised Sentence Compressionmentioning

confidence: 99%

Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning

Ghalandari¹,

Hokamp²,

Ifrim³

2022

Preprint

View full text Add to dashboard Cite

Sentence compression reduces the length of text by removing non-essential content while preserving important facts and grammaticality. Unsupervised objective driven methods for sentence compression can be used to create customized models without the need for ground-truth training data, while allowing flexibility in the objective function(s) that are used for learning and inference. Recent unsupervised sentence compression approaches use custom objectives to guide discrete search; however, guided search is expensive at inference time. In this work, we explore the use of reinforcement learning to train effective sentence compression models that are also fast when generating predictions. In particular, we cast the task as binary sequence labelling and fine-tune a pre-trained transformer using a simple policy gradient approach. Our approach outperforms other unsupervised models while also being more efficient at inference time.

show abstract

“…Sentence compression is currently dominated by supervised methods (Malireddy et al, 2020;Nguyen et al, 2020;Nóbrega et al, 2020) and highly relies on syntactic dependency trees (Le et al, 2019;Xu and Durrett, 2019b;Wang and Chen, 2019;Kamigaito and Okumura, 2020). Unsupervised methods have been explored to extract sentences from documents to represent the key points (Jang and Kang, 2021).…”

Section: Related Workmentioning

confidence: 99%

A Novel Metric for Evaluating Semantics Preservation

Peng¹,

Li²,

Zhao³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we leverage pre-trained language models (PLMs) to precisely evaluate the semantics preservation of edition process on sentences. Our metric, Neighbor Distribution Divergence (NDD), evaluates the disturbance on predicted distribution of neighboring words from mask language model (MLM). NDD is capable of detecting precise changes in semantics which are easily ignored by text similarity. By exploiting the property of NDD, we implement a unsupervised and even trainingfree algorithm for extractive sentence compression. We show that our NDD-based algorithm outperforms previous perplexity-based unsupervised algorithm by a large margin. For further exploration on interpretability, we evaluate NDD by pruning on syntactic dependency treebanks and apply NDD for predicate detection as well.

show abstract

SCAR: Sentence Compression using Autoencoders for Reconstruction

Cited by 7 publications

References 14 publications

Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model

Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model

Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning

A Novel Metric for Evaluating Semantics Preservation

Contact Info

Product

Resources

About