Evaluating Deep Learning Methods for Word Segmentation of Scripta Continua Texts in Old French and Latin

Abstract

International audience Tokenization of modern and old Western European languages seems to be fairly simple, as it stands on the presence mostly of markers such as spaces and punctuation. However, when dealing with old sources like manuscripts written in scripta continua, antiquity epigraphy or Middle Age manuscripts, (1) such markers are mostly absent, (2) spelling variation and rich morphology make dictionary based approaches difficult. Applying convolutional encoding to characters followed by linear categorization to word-boundary or in-word-sequence is shown to be effective at tokenizing such inputs. Additionally, the software is released with a simple interface for tokenizing a corpus or generating a training set.

Artificial colorization of digitized microfilms: a preliminary study

“…It must be stressed that spaces are the most important source of error in medieval HTR models 21 : for the model Bicerin (Pinche [2021a]), spaces represent 33.9% of errors 22 ). In the current state of the art of HTR, some workflows (Camps et al [2021(Camps et al [ , 2020) chose to solve this problem with a secondary tool such as Boudams (Clérice [2019]), a deep learning tool built for word segmentation in Latin or Medieval French. Of these, the microfilmed manuscripts (see Table 2) all dating from the end of the 13 th century or the 14 th century and written in Old French are kept for evaluating performances as our test dataset 23 .…”

EpiSearch. Identifying Ancient Inscriptions in Epigraphic Manuscripts

“…For the Latin inscriptions, transcribed by Astori in upper-case letters, we have used the Modèle imprimé 16-18e Fra+Lat. The model comes from the CREMMA project and combines French and Latin training data (such as that in this repository: [Clérice, 2021]).…”

RASAM – A Dataset for the Recognition and Analysis of Scripts in Arabic Maghrebi

“…Following the competitions organized in recent years, at ICFHR and ICDAR notably, several robust architectures for layout analysis of historical documents have been developed [8], whose application to non-Latin script documents provide equivalent results [10,14]. The HTR architectures specialized on a type of document or on a hand also achieve a very high recognition score, even though the literature is mostly Latin script based, as well as the proven pipelines composed of character-level HTR and post-processing [7]. The non-Latin, cursive and RTL writings, like the Arabic scripts, remain an open problem in digital humanities with a wide variety of approaches [11].…”

“…Therefore, the Maghrebi scripts constitute a family of rounded scripts that share a number of characteristics, first of all very rounded loops, that can be seen in the manuscripts in the present dataset (see infra 2.3). The main characteristics 7 of the scripts are displayed in table 1.…”

“…The most recent works, in particular those of U. Bongianino, have foregrounded the different itineraries (from books to qurans, from al-Andalus to the Maghreb) followed by these writings between the 10th and the 13th century [4]. 7 Characteristics are taken from U. Bongianino [4]; theoretical realizations are taken from the article of N. Van de Boogert upon which U. Bongianino draws [13].…”