Tilde’s Machine Translation Systems for WMT 2018

Abstract

The paper describes the development process of the Tilde's NMT systems that were submitted for the WMT 2018 shared task on news translation. We describe the data filtering and pre-processing workflows, the NMT system training architectures, and automatic evaluation results. For the WMT 2018 shared task, we submitted seven systems (both constrained and unconstrained) for English-Estonian and Estonian-English translation directions. The submitted systems were trained using Transformer models.

Findings of the 2018 Conference on Machine Translation (WMT18)

“…An additional system paper (Hu et al, 2018) describes a non-primary submission. (Pinnis et al, 2018) TILDE submitted four systems: TILDE-C-NMT, TILDE-C-NMT-COMB, TILDE-C-NMT-2BT and TILDE-NC-NMT. TILDE-C-NMT are constrained English-Estonian and Estonian-English NMT systems that were deployed as ensembles of averaged factored data Transformer models.…”

Tilde’s Machine Translation Systems for WMT 2019

“…This year, we did not change the parallel and monolingual data pre-processing workflows that we used for our WMT 2018 submissions (Pinnis et al, 2018a).…”

“…In our submissions for WMT 2018, we introduced an automatic named entity (NE) postediting (ANEPE) workflow (Pinnis et al, 2018a), which allowed to fix translations of NEs (consisting of one word) and non-translatable words after NMT decoding. The method depends on the quality of word alignments.…”

“…In our experiments, we investigated methods for corpora filtering (the Tilde MT parallel data filtering (TMTF) and normalisation workflow (Pinnis, 2018) together with dual conditional cross-entropy filtering (DCCEF) (Junczys-Dowmunt, 2018)), training data pre-processing using the methods described by Pinnis et al (2018a), a new optimisation method, the quasi-hyperbolic Adam, proposed by Ma and Yarats (2018), back-translation with sampling-based decoding (e.g., as done by ) and by targeting rare words (Fadaee and Monz, 2018) and in-domain subsets of the monolingual data, and automatic linguistically informed post-editing of named entities and non-translatable phrases.…”

Debugging Translations of Transformer-based Neural Machine Translation Systems

“…Aside from visualising and interpreting NMT output, attention alignments are also used to get hard word alignments in order to correctly translate structured documents and reconstruct the structure after translating (Pinnis et al, 2018b). To achieve similar results with transformer-based NMT models, several approaches have been explored, such as learning guided alignments 5 , averaging attention matrices 6 and using fast align (Dyer et al, 2013) to generate alignments after the translation has been produced (e.g., Pinnis et al (2018a)). The latter approach is of no use for interpreting NMT output as it uses a separate model and only attempts to guess what the alignments are after the result has been produced.…”