Filtering of Noisy Parallel Corpora Based on Hypothesis Generation

Abstract

The filtering task of noisy parallel corpora in WMT2019 aims to challenge participants to create filtering methods to be useful for training machine translation systems. In this work, we introduce a noisy parallel corpora filtering system based on generating hypotheses by means of a translation model. We train translation models in both language pairs: Nepali-English and Sinhala-English using provided parallel corpora. To create the best possible translation model, we first join all provided parallel corpora (Nepali, Sinhala and Hindi to English) and after that, we applied bilingual cross-entropy selection for both language pairs (Nepali-English and Sinhala-English). Once the translation models are trained, we translate the noisy corpora and generate a hypothesis for each sentence pair. We compute the smoothed BLEU score between the target sentence and generated hypothesis. In addition, we apply several rules to discard very noisy or inadequate sentences which can lower the translation score. These heuristics are based on sentence length, source and target similarity and source language detection. We compare our results with the baseline published on the shared task website, which uses the Zipporah model, over which we achieve significant improvements in one of the conditions in the shared task. The designed filtering system is domain independent and all experiments are conducted using neural machine translation.

Findings of the WMT 2019 Shared Task on Parallel Corpus Filtering for Low-Resource Conditions

“…Another approach is to first train a translation system on the clean data, then use it to translate the non-English side into English and use monolingual matching methods to compare it against the English side of the parallel corpus. Different matching metrics were used: METEOR (Erdmann and Gwinnup, 2019), Levenshtein distance (Sen et al, 2019), or BLEU (Parcheta et al, 2019), Several submissions considered vocabulary coverage in their methods, preferring to add sentence pairs to the limited set that increase the number of words and n-grams covered (Erdmann and Gwinnup, 2019;Bernier-Colborne and Lo, 2019;González-Rubio, 2019).…”

“…building neural machine translation models on the clean data and considering the translation scores from forced translation of the parallel corpus. One submission used this method , while others applied the same idea to monolingual language model scores (Axelrod, 2019;Parcheta et al, 2019).…”

ParaCrawl: Web-Scale Acquisition of Parallel Corpora

“…Another approach is to first train a translation system on the clean data, then use it to translate the non-English side into English and use monolingual matching methods to compare it against the English side of the parallel corpus. Different matching metrics were used: METEOR (Erdmann and Gwinnup, 2019), Levenshtein distance (Sen et al, 2019), or BLEU (Parcheta et al, 2019),…”

Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora

“…Chaudhary et al (2019) propose the use of cross-lingual sentence embeddings for determining sentence pair quality while several efforts (Kurfalı and Östling, 2019;Soares and Costajussà, 2019;Bernier-Colborne and Lo, 2019) have focused on the use of monolingual word embeddings. Parcheta et al (2019) use a machine translation system trained on clean data to translate the source sentences of the noisy corpus and evaluate the translation against the original target sentences using BLEU scores. Erdmann and Gwinnup (2019) and Sen et al (2019) propose similar methods using METEOR scores and Levenshtein distance respectively.…”