Improved statistical machine translation using paraphrases

Abstract

Parallel corpora are crucial for training SMT systems. However, for many language pairs they are available only in very limited quantities. For these language pairs a huge portion of phrases encountered at run-time will be unknown. We show how techniques from paraphrasing can be used to deal with these otherwise unknown source language phrases. Our results show that augmenting a stateof-the-art SMT system with paraphrases leads to significantly improved coverage and translation quality. For a training corpus with 10,000 sentence pairs we increase the coverage of unique test set unigrams from 48% to 90%, with more than half of the newly covered items accurately translated, as opposed to none in current approaches.

Building Subjectivity Lexicon(s) from Scratch for Essay Data

“…In particular, it has been used to improve several tasks such as query expansion in Information Retrieval [36][37], evaluation of NLP systems [38][39][40] and statistical machine translation [41,42]. A more comprehensive review of paraphrase generation and its applications can be found in [43].…”

Japanese Argument Reordering Based on Dependency Structure for Statistical Machine Translation

“…5 concludes the paper and suggests future work. [7] used paraphrases only when there was an unknown word or phrase. They extracted the unknown word/phrase, found equivalent paraphrases and then added the translation pair into the phrase table.…”

Using Verbs to Characterize Noun-Noun Relations

“…Example Extracted Verbs AGENT student protest be led by (6), be sponsored by(6), pit(4), be(4), be organized by(3), be staged by(3), be launched by(3), be started by(3), be supported by(3), involve(3), arise from(3) AGENT band concert feature(17), capture(10), include(6), be given by(6), play of(4), involve(4), be than(4) be organized by(3), be by(3), start with(3), bring(3), take(3), consist of(3) AGENT military assault be initiated by(4), shatter (2) BENEFICIARY student price be (14), mean(4), differ from(4), be unfair for(3), be discounted for(3), be for(3), be affordable for(3), be charged for(3), please(3), be shared with(3), draw in(3) CAUSE exam anxiety be generated during (3) CONTAINER printer tray hold (12), come with(9), be folded (8), fit under(6), be folded into(4), pull from(4), be inserted into(4), be mounted on(4), be used by(4), be inside(3), feed into(3) CONTAINER flood water cause (24), produce(9), remain after (9), be swept by(6), create(5), bring(5), reinforce(5) CONTAINER film music fit (16), be in(13), be used in(11), be heard in (11), play throughout(9), be written for(9) CONTAINER story idea tell (20), make (19), drive(15), become(13), turn into(12), underlie (12), occur within(8), hold(8), tie(8), be(8), spark (8), appear throughout(7), tell(7), move(7), come from(6) CONTENT paper tray feed(6), be lined with(6), stand up(6), hold(4), contain(4), catch(4), overflow with(3) CONTENT eviction notice result in(10), precede(3), make(2)…”

“…For example, they may help in the process of noun compound translation [23]. They could be also directly integrated into a paraphrase-augmented machine translation system [24], machine translation evaluation system [25] [26], or summarization evaluation system [27].…”