Japanese–English Conversation Parallel Corpus for Promoting Context-aware Machine Translation Research

Abstract

Most machine translation (MT) research has focused on sentences as translation units (sentence-level MT), and has achieved acceptable translation quality for sentences where cross-sentential context is not required in mainly high-resourced languages. Recently, many researchers have worked on MT models that can consider a crosssentential context. These models are often called context-aware MT or documentlevel MT models. Document-level MT is difficult to 1) train with a small amount of document-level data; and 2) evaluate, as the main methods and datasets focus on sentence-level evaluation. To address the first issue, we present a Japanese-English conversation corpus in which the cross-sentential context is available. As for the second issue, we manually identify the main areas where sentence-level MT fails to produce adequate translations in the lack of context. We then create an evaluation set in which these phenomena are annotated to alleviate the automatic evaluation of document-level systems. We train MT models using our corpus to demonstrate how the use of context leads to improvements.

TANDO^+: Corpus and Baselines for Document-level Machine Translation in Basque-Spanish and Basque-French

“…For instance, some improvements obtained with context-aware models, as measured by standard translation metrics, may be attributed to context-driven regularisation acting as a noise generator, particularly with small-scale data (Kim et al, 2019;Li et al, 2020). Nonetheless, several studies have established that context information can indeed be effectively modelled to tackle discursive phenomena in NMT beyond the sentence level (Liu and Zhang, 2020;Rikters and Nakazawa, 2021;Xu et al, 2021;Mansimov et al, 2021;Gete et al, 2023).…”

“…Context modelling in NMT is typically performed over a fixed window of preceding or following context sentences (Zhang et al, 2018;Voita et al, 2018Voita et al, , 2019bYang et al, 2019;Rikters and Nakazawa, 2021). This is due in part to the challenges associated with long-distance context modelling, resulting in degraded translation quality on long-range context (Junczys-Dowmunt, 2019; Tan et al, 2019;Zheng et al, 2020;Sun et al, 2020).…”

Machine Translation of Scheduling Joint Optimization Algorithm in Japanese Passive Statistics

“…In the Japanese passive statistical machine translation model, the corresponding passive parameters and the parameters to be translated are trained to form the search results with the maximum probability. If different types of passives are decoded, the optimal path for translation is finally formed [12].…”