Response-based Learning for Grounded Machine Translation

Abstract

We propose a novel learning approach for statistical machine translation (SMT) that allows to extract supervision signals for structured learning from an extrinsic response to a translation input. We show how to generate responses by grounding SMT in the task of executing a semantic parse of a translated query against a database. Experiments on the GEO-QUERY database show an improvement of about 6 points in F1-score for responsebased learning over learning from references only on returning the correct answer from a semantic parse of a translated query. In general, our approach alleviates the dependency on human reference translations and solves the reachability problem in structured learning for SMT.

A Corpus and Semantic Parser for Multilingual Natural Language Querying of OpenStreetMap

“…In a similar way, multilingual database access can be enhanced by adapting an SMT system by responsebased learning, using executability of a parse of a translated query as supervision signal (Riezler et al, 2014). Both cases of response-based learning only require a user who issues a query and gives feedback on whether the proposed OSM object was the intended answer.…”

“…Our SMT tuning experiment builds on the work of Riezler et al (2014) and Haas and Riezler (2015) who applied response-based learning for SMT to the GEOQUERY and FREE917 domains, respectively. …”

“…While both REBOL and RAMPION assume the availability of both a reference translation and a gold parse, response-based learning can also succeed without any access to reference translations or even to gold standard parses. Riezler et al (2014) introduced an algorithm, called EXEC, that only relies on task-based feedback and omits the cost function based on sentence-wise BLEU. Collecting real world data for this algorithm is realistic for an online interface to OSM since it only requires a user to pose a question and then indicate if it was answered to their satisfaction.…”

An Investigation of Machine Translation Evaluation Metrics in Cross-lingual Question Answering

“…While SEMPRE has shown highly competitive results on standard QA tasks, we also plan to examine other methods such as Berant and Liang (2014)'s semantic parsing through paraphrasing, which may be less sensitive to superficial differences in surface forms of the translation results. We also plan to to optimize machine translation systems using this analysis, possibly through incorporation into the responsebased learning framework of Riezler et al (2014).…”

“…For example, according to the work of Hyodo and Akiba ( 2009), a translation model trained using a parallel corpus without function words achieved higher accuracy than a model trained using full sentences on CLQA using documents or web pages, although it is not clear whether these results will apply to more structured QA using knowledge bases. There is also work on optimizing translation to improve CLQA accuracy (Riezler et al, 2014;Haas and Riezler, 2015), but these methods require a large set of translated questionanswer pairs, which may not be available in many languages. Correspondingly, it is of interest to investigate which factors of translation output affect CLQA accuracy, which is the first step towards designing MT systems that achieve better accuracy on the task.…”