Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation

Abstract

Understanding voluminous historical records provides clues on the past in various aspects, such as social and political issues and even natural science facts. However, it is generally difficult to fully utilize the historical records, since most of the documents are not written in a modern language and part of the contents are damaged over time. As a result, restoring the damaged or unrecognizable parts as well as translating the records into modern languages are crucial tasks. In response, we present a multi-task learning approach to restore and translate historical documents based on a selfattention mechanism, specifically utilizing two Korean historical records, ones of the most voluminous historical records in the world. Experimental results show that our approach significantly improves the accuracy of the translation task than baselines without multi-task learning. In addition, we present an in-depth exploratory analysis on our translated results via topic modeling, uncovering several significant historical events.

Translating Hanja historical documents to understandable Korean and English

“…There is a significant perplexity difference between the ground truth nKo translation and oKo, which means the gt-nKo translation is closer to the modern language than the gt-oKo. Our generated translations show a lower perplexity than the gt-oKo and Kang et al (2021); it is closer to the modern language similar to gt-nKo.…”

“…To translate AJD with the neural network, Park et al (2020) proposes a new subword tokenization method called the share vocabulary and entity restriction byte pair encoding. Kang et al (2021) presents a multi-task learning approach that restoring and translating the historical documents. For the restoration task, they used the untranslated Diaries of the Royal Secretariat (DRS) data, Korean historical documents written in Hanja.…”

“…Old Korean Translation For the comparison of our model to the previous approach, (Kang et al, 2021) that used oKo as the target language translations, we only used the 〈Hanja, oKo〉 corpus for training. Table 3 ( language of the additional data is Hanja, so the encoder learned richer Hanja representation compared to the oKo.…”

Restoring and attributing ancient texts using deep neural networks

“…Inspired by biological neural networks, deep neural networks can discover and harness intricate statistical patterns in vast quantities of data 10 . Recent increases in computational power have enabled these models to tackle challenges of growing sophistication in many fields [11][12][13][14] , including the study of ancient languages [15][16][17][18] .…”

“…The closest work to Ithaca is our 2019 research on ancient text restoration: Pythia 15 . Pythia was to our knowledge the first ancient text restoration model to use deep neural networks, and was followed by blank language models 18 , Babylonian 65 and Korean text translation and restoration 17 , Latin BERT for language modelling, part-of-speech tagging, word sense disambiguation and word similarity 16 , and the classification of Cuneiform tablets by period 66 .…”