Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices

Abstract

We present a novel scheme to combine neural machine translation (NMT) with traditional statistical machine translation (SMT). Our approach borrows ideas from linearised lattice minimum Bayes-risk decoding for SMT. The NMT score is combined with the Bayes-risk of the translation according the SMT lattice. This makes our approach much more flexible than n-best list or lattice rescoring as the neural decoder is not restricted to the SMT search space. We show an efficient and simple way to integrate risk estimation into the NMT decoder which is suitable for word-level as well as subword-unit-level NMT. We test our method on EnglishGerman and Japanese-English and report significant gains over lattice rescoring on several data sets for both single and ensembled NMT. The MBR decoder produces entirely new hypotheses far beyond simply rescoring the SMT search space or fixing UNKs in the NMT output.

The CUED’s Grammatical Error Correction Systems for BEA-2019

“…Finally, we map the word-level FSTs to the subword-level by composition with a mapping transducer T that applies byte pair encoding (Sennrich et al, 2016c, BPE) to the full words. Word-to-BPE mapping transducers have been used in prior work to combine word-level models with subword-level neural sequence models (Stahlberg et al, , 2017b(Stahlberg et al, , 2018b(Stahlberg et al, , 2017a.…”

SGNMT – A Flexible NMT Decoding Platform for Quick Prototyping of New Models and Search Strategies

“…Restricts the search space to a bag of words with or without repetition (Hasler et al, 2017 • consume(token) Update the internal predictor state by adding token to the current history.…”

“…• nmt,ngramc,wc: MBR-based NMT following Stahlberg et al (2017) with n-gram posteriors extracted from an SMT lattice (ngramc) and a simple word penalty (wc).…”

“…system) rather than only one as in normal beam search. We report a moderate gain of 0.5 BLEU over a single system on the Japanese-English ASPEC test set (Nakazawa et al, 2016) by combining three BPE-based NMT models from Stahlberg et al (2017) using the sepbeam decoder.…”

CUED@WMT19:EWC&LMs

“…• Even though the performance gap between NMT and traditional statistical machine translation (SMT) is growing rapidly on the task at hand, SMT can still improve very strong NMT ensembles. To combine NMT and SMT we follow Stahlberg et al (2017aStahlberg et al ( , 2018b and build a specialized n-gram LM for each sentence that computes the risk of hypotheses relative to SMT lattices.…”