Better Document-Level Machine Translation with Bayes’ Rule

Abstract

<jats:p> We show that Bayes’ rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents a compelling benefit because parallel documents are not always available. In our formulation, the posterior probability of a candidate translation is the product of the unconditional (prior) probability of the candidate output document and the “reverse translation probability” of translating the candidate output back into the source language. Our proposed model uses a powerful autoregressive language model as the prior on target language documents, but it assumes that each sentence is translated independently from the target to the source language. Crucially, at test time, when a source document is observed, the document language model prior induces dependencies between the translations of the source sentences in the posterior. The model’s independence assumption not only enables efficient use of available data, but it additionally admits a practical left-to-right beam-search algorithm for carrying out inference. Experiments show that our model benefits from using cross-sentence context in the language model, and it outperforms existing document translation approaches. </jats:p>

Data-to-Text Generation with Content Selection and Planning

“…In this case, attention diffuses over the relatively long MLB macro plan, leading to inaccurate content selection. We could alleviate this problem by adopting a noisy channel decomposition (Yee et al, 2019;Yu et al, 2020), i.e., by learning two different distributions: a conditional model which provides the probability of translating a paragraph plan to text and a language model which provides an unconditional estimate of the output (i.e., the whole game summary). However, we leave this to future work.…”

G-Transformer for Document-Level Machine Translation

“…Different from these approaches, G-Transformer uses a generic design for both source and context, translating whole document in one beam search instead of sentence-by-sentence. Some methods use a two-pass strategy, generating sentence translation first, integrating context information through a post-editing model (Voita et al, 2019a;Yu et al, 2020). In contrast, G-Transformer uses a single model, which reduces the complexity for both training and inference.…”

Capturing document context inside sentence-level neural machine translation models with self-training

“…Another line of document-level NMT work (Xiong et al, 2018;Voita et al, 2019b) proposed a twopass document decoding model inspired by the deliberation network (Xia et al, 2017) in order to incorporate target side document context. A parallel line of work (Garcia et al, 2017(Garcia et al, , 2019Yu et al, 2019) introduced document-level approaches that do not require training the context-conditional NMT model by introducing a separate language model to enforce the consistency in the outputs of sentence-level NMT model. Garcia et al (2019) used a simple n-gram based semantic space language model (Hardmeier et al, 2012) to re-rank the outputs of the sentence-level NMT model inside the beam-search algorithm to enforce documentlevel consistency.…”