Ancient Korean Neural Machine Translation

Abstract

Translation of the languages of ancient times can serve as a source for the content of various digital media and can be helpful in various fields such as natural phenomena, medicine, and science. Owing to these needs, there has been a global movement to translate ancient languages, but expert minds are required for this purpose. It is difficult to train language experts, and more importantly, manual translation is a slow process. Consequently, the recovery of ancient characters using machine translation has been recently investigated, but there is currently no literature on the machine translation of ancient Korean. This paper proposes the first ancient Korean neural machine translation model using a Transformer. This model can improve the efficiency of a translator by quickly providing a draft translation for a number of untranslated ancient documents. Furthermore, a new subword tokenization method called the Share Vocabulary and Entity Restriction Byte Pair Encoding is proposed based on the characteristics of ancient Korean sentences. This proposed method yields an increase in the performance of the original conventional subword tokenization methods such as byte pair encoding by 5.25 BLEU points. In addition, various decoding strategies such as n-gram blocking and ensemble models further improve the performance by 2.89 BLEU points. The model has been made publicly available as a software application.

Time-Aware Ancient Chinese Text Translation and Inference

“…Therefore, we direct our efforts toward three particular eras: Pre-Qin (先秦), Han (汉), and Song (宋) to verify the hypothesis that the chronology of a text directly influences the word meaning and model performance. In particular, Pre-Qin and Han are closer chronologically, so we expect their model performances to be closer than that between Pre-Qin and Song, as was shown in other ancient text translation (Park et al, 2020). One reason for this difference is the use of polysemous single-character words, which are highly ambiguous.…”

A Transformer-Based Neural Machine Translation Model for Arabic Dialects That Utilizes Subword Units

“…This research study employed 21,000-subword vocabularies for Levantine vernacular (LEV)—MSA translation task, 21,000 subword vocabularies for Maghrebi vernacular (MAG)–MSA translation task, 21,000 subword vocabularies for the Nile Basin Arabic (NB)—MSA translation task, 21,000 subword vocabularies for the Gulf Arabic (Gulf)—MSA translation task and 9235 subword vocabularies for the Iraqi Arabic (IRQ)—MSA translation task. Twenty-nine thousand five hundred subword vocabularies are employed for the corpus applied by Baniata et al [ 24 ] on the Levantine Arabic (LEV)–MSA translation task and on the Maghrebi vernacular (MAG)–MSA translation task. Relu dropout value and attention dropout value are 0.1.…”

“…These experimental findings showed that the usage of UTagger and RDRsegmenter in the Korean–Vietnamese NMT system might increase its performance, obtaining exceptional outcomes from Korean to Vietnamese with BLEU score of 27.79 and TER score of 58.77 and in reverse way a BLEU score of 25.44 and TER score of 58.72. Park et al [ 24 ] proposed the first ancient Korean NMT system based on the use of a Transformer. The method improves translator performance by instantly generating a draft translation for different ancient documents that remain untranslated.…”

Translation Mechanism of Neural Machine Algorithm for Online English Resources

“…It is a translation method based on a parallel corpus of source and target languages [5]. Aiming at the locality problem of statistical machine translation, the architecture of neural machine translation adopts the overall sentence-to-sentence translation, which can obtain longer-distance dependencies as much as possible; for the problem of the complicated process of statistical machine translation and numerous functional components, the neural machine translation system is single in the structure; that is, a complete neural network structure can complete the entire translation process and does not need to consider the cooperation and coordination between various components like statistical machine translation [6,7]. Because neural networks can automatically capture useful features in data, neural machine translation also avoids complicated manual design features.…”