Paraphrasing with bilingual parallel corpora

Abstract

Previous work has used monolingual parallel corpora to extract and generate paraphrases. We show that this task can be done using bilingual parallel corpora, a much more commonly available resource. Using alignment techniques from phrasebased statistical machine translation, we show how paraphrases in one language can be identified using a phrase in another language as a pivot. We define a paraphrase probability that allows paraphrases extracted from a bilingual parallel corpus to be ranked using translation probabilities, and show how it can be refined to take contextual information into account. We evaluate our paraphrase extraction and ranking methods using a set of manual word alignments, and contrast the quality with paraphrases extracted from automatic alignments.

Simplification Using Paraphrases and Context-Based Lexical Substitution

“…PPDB is a collection of more than 100 million English paraphrase pairs (Ganitkevitch et al, 2013). These pairs were extracted using a bilingual pivoting technique (Bannard and Callison-Burch, 2005), which assumes that two English phrases that translate to the same foreign phrase have the same meaning. PPDB was updated by Pavlick et al (2015) to assign labels stating the precise entailment relationship between paraphrase pairs (e.g.…”

Building Subjectivity Lexicon(s) from Scratch for Essay Data

“…This system operates by extracting bilingual correspondences from a bilingual corpus, identifying phrases and words in Language A that all correspond to the same phrase or word in Language B and pivoting on the common phrase or word to extract all Language A words and phrases as paraphrases of each other. More details can be found in [7]. We use the French-English parallel corpus (approx.…”

Japanese Argument Reordering Based on Dependency Structure for Statistical Machine Translation

“…Then, they translated the sentence using the new phrase table. The paraphrases were generated using bilingual parallel corpora where phrases were extracted when a single source phrase was translated into multiple target phrases [8]. While this method may suffer from errors when substituting the paraphrases into the sentence, syntactic constraints were applied later to improve the results [9].…”

“…However, one can also use paraphrases as the input source without changing the translation model. [12] generated a paraphrase lattice for each sentence by building a lattice based on the paraphrase model [8]. Translation is carried out by lattice decoding over the paraphrase lattice with additional features: paraphrase probability, language model score and paraphrase length.…”