Neural Machine Translation with Adequacy-Oriented Learning

Abstract

Although Neural Machine Translation (NMT) models have advanced state-of-the-art performance in machine translation, they face problems like the inadequate translation. We attribute this to that the standard Maximum Likelihood Estimation (MLE) cannot judge the real translation quality due to its several limitations. In this work, we propose an adequacyoriented learning mechanism for NMT by casting translation as a stochastic policy in Reinforcement Learning (RL), where the reward is estimated by explicitly measuring translation adequacy. Benefiting from the sequence-level training of RL strategy and a more accurate reward designed specifically for translation, our model outperforms multiple strong baselines, including (1) standard and coverage-augmented attention models with MLE-based training, and (2) advanced reinforcement and adversarial training strategies with rewards based on both word-level BLEU and character-level CHRF3. Quantitative and qualitative analyses on different language pairs and NMT architectures demonstrate the effectiveness and universality of the proposed approach.

Dynamic Past and Future for Neural Machine Translation

“…We also list the results of other published research for comparison, where our model outperforms the previous results in both language pairs. Note that our approach also surpasses Kong et al (2019) on WMT14 En-De task. These experiments demonstrate the effectiveness of our approach across different language pairs.…”

“…Ideally, with all the source encoded representations in the encoder, NMT models should be able to update translated and untranslated source contents and keep them in mind. However, most of existing NMT models lack an explicit functionality to maintain the translated and untranslated contents, failing to distinguish the source words being of either PAST or FUTURE (Zheng et al, 2018), which is likely to suffer from severe inadequate translation problem (Tu et al, 2016;Kong et al, 2019).…”

“…We list the results of our experiments on NIST Zh-En task in Table 1 (Wu et al, 2016) 24.6 N/A ConvS2S (Gehring et al, 2017) 25.2 29.88 Transformer (Vaswani et al, 2017) 27.3 N/A +AOL (Kong et al, 2019) 28.01 N/A Transformer (Gu et al, 2017) N/A 31.91 Transformer 27.14 32.10 OURS 28.10 32.96 which fails to be compatible with parallel training of Transformer, scarifying Transformer's efficiency advantage.…”

“…Inadequate translation problem is a widely known weakness of NMT models, especially when translating long sentences (Kong et al, 2019;Tu et al, 2016;. To alleviate this problem, one direction is to recognize the translated and untranslated contents, and pay more attention to untranslated parts.…”

“…The PAST capsules achieves r P OL of 0.72, while r F OL of 0.70 for the FUTURE capsules. The results indicate that the capsules could predict the corresponding words to a certain extent, which implies the capsules contain the expected information of PAST or FUTURE contents.Translations become better and more adequate.To validate the translation adequacy of our model, we use Coverage Difference Ratio (CDR) proposed byKong et al (2019), i.e., CDR = 1 −|C ref \Cgen| |C ref |, where C ref and C gen are the set of source words covered by the reference and translation, respectively. The CDR reflects the translation adequacy by comparing the source coverages be-…”