Restoring ancient text using deep learning: a case study on Greek epigraphy

Abstract

Ancient History relies on disciplines such as Epigraphy, the study of ancient inscribed texts, for evidence of the recorded past. However, these texts, "inscriptions", are often damaged over the centuries, and illegible parts of the text must be restored by specialists, known as epigraphists. This work presents PYTHIA, the first ancient text restoration model that recovers missing characters from a damaged text input using deep neural networks. Its architecture is carefully designed to handle longterm context information, and deal efficiently with missing or corrupted character and word representations. To train it, we wrote a nontrivial pipeline to convert PHI, the largest digital corpus of ancient Greek inscriptions, to machine actionable text, which we call PHI-ML. On PHI-ML, PYTHIA's predictions achieve a 30.1% character error rate, compared to the 57.3% of human epigraphists. Moreover, in 73.5% of cases the ground-truth sequence was among the Top-20 hypotheses of PYTHIA, which effectively demonstrates the impact of this assistive method on the field of digital epigraphy, and sets the state-of-the-art in ancient text restoration.

Blank Language Models

“…We test BLM's capacity to rewrite specified portions of text on three tasks: text infilling (Zhu et al, 2019), ancient text restoration (Assael et al, 2019) and style transfer (Shen et al, 2017). Fig.…”

Restoration of fragmentary Babylonian texts using recurrent neural networks

“…The best results in solving the restoration task, so far, have been achieved in studies of machine-reading comprehension, specifically of the cloze-style, in which both the level of character and word are identified (45). In the field of ancient languages, we know of only one other study that used an algorithm with neural network architecture to recover missing letters, in the context of epigraphic inscriptions in ancient Greek (46). Their model, named PYTHIA, uses a sequence-to-sequence NN with LSTM and was trained on the Packard Humanities Institute (PHI) database, the largest digital dataset of ancient Greek inscriptions.…”

Ancient Korean Neural Machine Translation

“…It supports detoxification or automatic conversion of the text to Chinese characters or Hiragana, providing an opportunity to assist the interpretation of the vast amount of historical text that has remained unconverted. Oxford and DeepMind conducted an initial study to restore ancient Greek letters using deep learning [1]. They proposed PYTHIA, the ancient text restoration model that recovers missing characters from a damaged input text using deep learning.…”

“…Currently, there are no existing studies on ancient Korean machine translation. However, machine translation studies for other ancient languages are starting to appear, such as KuroNET [4,5] for ancient Japanese and research on ancient Greek epigraphies [1]. Hence, this study is the first to investigate ancient Korean machine translation and will serve as a starting point for further ancient Korean machine translation research.…”

“…• A software application based on the proposed ancient translator system was created and officially distributed. 1 Using this application, anyone can perform ancient translation, and it can provide draft translations to human experts.…”