UAlacant word-level machine translation quality estimation system at WMT 2015

Abstract

This paper describes the Universitat d'Alacant submissions (labelled as UAlacant) for the machine translation quality estimation (MTQE) shared task in WMT 2015, where we participated in the wordlevel MTQE sub-task. The method we used to produce our submissions uses external sources of bilingual information as a black box to spot sub-segment correspondences between a source segment S and the translation hypothesis T produced by a machine translation system. This is done by segmenting both S and T into overlapping subsegments of variable length and translating them in both translation directions, using the available sources of bilingual information on the fly. For our submissions, two sources of bilingual information were used: machine translation (Apertium and Google Translate) and the bilingual concordancer Reverso Context. After obtaining the subsegment correspondences, a collection of features is extracted from them, which are then used by a binary classifer to obtain the final "GOOD" or "BAD" word-level quality labels. We prepared two submissions for this year's edition of WMT 2015: one using the features produced by our system, and one combining them with the baseline features published by the organisers of the task, which were ranked third and first for the sub-task, respectively.

UAlacant word-level and phrase-level machine translation quality estimation systems at WMT 2016

“…The table also includes the results obtained with a binary classifier trained only on the baseline features (baseline), in order to estimate the contribution of the features described in this work on the performance of the system. Incidentally, and in spite of the changes in languages and machine translation systems, the results obtained for word-level MTQE are very similar to those obtained by Esplà-Gomis et al (2015) for the translation from English into Spanish.…”

“…dcs.shef.ac.uk/wmt16_files_qe/task2_ en-de_test.tar.gz 6 The list of features can be found in the file features list in the package http://www.quest. dcs.shef.ac.uk/wmt16_files_qe/task2p_ en-de_test.tar.gz 7 The rest of parameters of the classifiers were also kept as in the approach byEsplà-Gomis et al (2015).…”

Predicting insertion positions in word-level machine translation quality estimation

“…The method described in this paper is based on previous approaches by the same authors [14,13], which are in turn based on the work by Esplà-Gomis et al [36], in which several online MT systems were used for word-level QE in translation-memory-based computer-aided translation tasks. The objective is for the method to be system-independent and able to use available online bilingual resources for word-level MT QE.…”

“…We have used two different kinds of sources of bilingual information: MT, a less informative bilingual resource (M ), and a bilingual concordancer, a more informative resource that provides the number of occurrences of each sub-segment translation (M occ ). We used three MT systems that are freely available on the Internet: Apertium [43], Lucy, 13 and Google Translate. 14 While Google Translate was used for both datasets, Lucy was used only for WMT16 and Apertium for WMT15.…”

“…The approach presented here builds on previous work by the same authors [13,14] in which insertion positions were not detected and a slightly different feature set was used. As in the original papers, here we use black-box bilingual resources from the Internet.…”