LIG System for Word Level QE task at WMT14

Abstract

This paper describes our Word-level QE system for WMT 2014 shared task on Spanish -English pair. Compared to WMT 2013, this year's task is different due to the lack of SMT setting information and additional resources.We report how we overcome this challenge to retain most of the important features which performed well last year in our system. Novel features related to the availability of multiple systems output (new point of this year) are also proposed and experimented along with baseline set. The system is optimized by several ways: tuning the classification threshold, combining with WMT 2013 data, and refining using Feature Selection strategy on our development set, before dealing with the test set for submission.

Multi-level Translation Quality Prediction with QuEst++

“…We explore a range of features from recent work (Bicici and Way, 2014;Camargo de Souza et al, 2014;Luong et al, 2014;Wisniewski et al, 2014), totalling 40 features of seven types:…”

“…Word-level QE (Blatz et al, 2004;Luong et al, 2014) is seemingly a more challenging task where a quality label is to be produced for each target word. An additional challenge is the acquisition of sizable training sets.…”

LORIA System for the WMT15 Quality Estimation Shared Task

“…(Luong et al, 2014) decide of the correctness of each word in t by checking its presence in two pseudo-references. The binary feature is based on the number of pseudo-references containing the evaluated word.…”

“…(Wisniewski et al, 2014) do not precise the number of pseudo-references, but they use the lattice produced by their in-house system, this leads certainly to a high number of pseudoreferences. (Luong et al, 2014;Wisniewski et al, 2014) works are applied to word-level Quality Estimation while we deal with sentence-level Quality Estimation. (Scarton and Specia, 2014) use features from pseudo-reference sentences for sentence-level quality estimation.…”

Unbabel's Participation in the WMT16 Word-Level Translation Quality Estimation Shared Task

“…QUETCH+ predicts the labels of individual words by combining a linear feature-based classifier with a feedforward neural network (called QUETCH, for QUality Estimation from scraTCH). The linear classifier is based upon Luong et al (2014) and uses the baseline features provided in the shared task. The QUETCH neural network is a multilayer perceptron, which takes as input the embeddings of the target words and the aligned source words, along with their context, and outputs a binary label for the target word.…”