A Comparison of Neural Models for Word Ordering

Hasler, Eva; Stahlberg, Felix; Tomalin, Marcus; Gispert, Adrià de; Byrne, Bill

doi:10.18653/v1/w17-3531

Cited by 12 publications

(30 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case of the Decoder, we used two layers and the attention mechanism proposed by Bahdanau et al (2014) in order to consider all words in the contexts (due to the unordered words). This proposal was similar to the recurrent neural network language model proposed in (Hasler et al, 2017).…”

Section: Word Orderingmentioning

confidence: 77%

“…Our proposal was motivated by the works of (Hasler et al, 2017) and (Zhang and Clark, 2015). Thus, we tackled the problem by applying a syntax-based word ordering strategy using a sequence-to-sequence model (seq-2-seq).…”

Section: System Descriptionmentioning

confidence: 99%

“…In relation to this problem, this has been addressed using language modeling (Schmaltz et al, 2016) and syntax-based approaches (Zhang and Clark, 2015). Recently, sequence-to-sequence models have also been used to tackle this problem, obtaining good results (Hasler et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

NILC-SWORNEMO at the Surface Realization Shared Task: Exploring Syntax-Based Word Ordering using Neural Models

Cabezudo¹,

Pardo²

2018

Proceedings of the First Workshop on Multilingual Surface Realisation

View full text Add to dashboard Cite

This paper describes the submission by the NILC Computational Linguistics research group of the University of São Paulo/Brazil to the Track 1 of the Surface Realization Shared Task (SRST Track 1). We present a neuralbased method that works at the syntactic level to order the words (which we refer by NILC-SWORNEMO, standing for "Syntax-based Word ORdering using NEural MOdels"). Additionally, we apply a bottom-up approach to build the sentence and, using language-specific lexicons, we produce the proper word form of each lemma in the sentence. The results obtained by our method outperformed the average of the results for English, Portuguese and Spanish in the track.

show abstract

Section: Word Orderingmentioning

confidence: 77%

Section: System Descriptionmentioning

confidence: 99%

See 1 more Smart Citation

NILC-SWORNEMO at the Surface Realization Shared Task: Exploring Syntax-Based Word Ordering using Neural Models

Cabezudo¹,

Pardo²

2018

Proceedings of the First Workshop on Multilingual Surface Realisation

View full text Add to dashboard Cite

show abstract

“…One way to achieve this goal would be to reconstruct readable documents from the bag-of-words output that our mechanism currently provides. A range of promising techniques for reconstructing readable texts from bag-of-words have already produced some good experimental results [19,52,54]. In future work we aim to explore how techniques such as these could be applied as a final post processing step for our mechanism.…”

Section: Discussionmentioning

confidence: 99%

Generalised Differential Privacy for Text Document Processing

Fernandes

Dras

McIver

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We address the problem of how to "obfuscate" texts by removing stylistic clues which can identify authorship, whilst preserving (as much as possible) the content of the text. In this paper we combine ideas from "generalised differential privacy" and machine learning techniques for text processing to model privacy for text documents. We define a privacy mechanism that operates at the level of text documents represented as "bags-of-words" -these representations are typical in machine learning and contain sufficient information to carry out many kinds of classification tasks including topic identification and authorship attribution (of the original documents). We show that our mechanism satisfies privacy with respect to a metric for semantic similarity, thereby providing a balance between utility, defined by the semantic content of texts, with the obfuscation of stylistic clues. We demonstrate our implementation on a "fan fiction" dataset, confirming that it is indeed possible to disguise writing style effectively whilst preserving enough information and variation for accurate content classification tasks.

show abstract

“…Evaluation Metrics: Linearisation tasks are generally reported using BLEU (Papineni et al, 2002) score (Hasler et al, 2017;Belz et al, 2011). Additionally, we report Kendall's Tau (τ ) and perfect match scores for the models.…”

Section: Methodsmentioning

confidence: 99%

Poetry to Prose Conversion in Sanskrit as a Linearisation Task: A Case for Low-Resource Languages

Krishna

Sharma

Santra

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

The word ordering in a Sanskrit verse is often not aligned with its corresponding prose order. Conversion of the verse to its corresponding prose helps in better comprehension of the construction. Owing to the resource constraints, we formulate this task as a word ordering (linearisation) task. In doing so, we completely ignore the word arrangement at the verse side. kāvya guru, the approach we propose, essentially consists of a pipeline of two pretraining steps followed by a seq2seq model. The first pretraining step learns task specific token embeddings from pretrained embeddings. In the next step, we generate multiple hypotheses for possible word arrangements of the input (Wang et al., 2018). We then use them as inputs to a neural seq2seq model for the final prediction. We empirically show that the hypotheses generated by our pretraining step result in predictions that consistently outperform predictions based on the original order in the verse. Overall, kāvya guru outperforms current state of the art models in linearisation for the poetry to prose conversion task in Sanskrit.

show abstract

A Comparison of Neural Models for Word Ordering

Cited by 12 publications

References 12 publications

NILC-SWORNEMO at the Surface Realization Shared Task: Exploring Syntax-Based Word Ordering using Neural Models

NILC-SWORNEMO at the Surface Realization Shared Task: Exploring Syntax-Based Word Ordering using Neural Models

Generalised Differential Privacy for Text Document Processing

Poetry to Prose Conversion in Sanskrit as a Linearisation Task: A Case for Low-Resource Languages

Contact Info

Product

Resources

About