Identifying Temporal Trends Based on Perplexity and Clustering: Are We Looking at Language Change?

Abstract

In this work we propose a data-driven methodology for identifying temporal trends in a corpus of medieval charters. We have used perplexities derived from RNNs as a distance measure between documents and then, performed clustering on those distances. We argue that perplexities calculated by such language models are representative of temporal trends. The clusters produced using the K-Means algorithm give an insight of the differences in language in different time periods at least partly due to language change. We suggest that the temporal distribution of the individual clusters might provide a more nuanced picture of temporal trends compared to discrete bins, thus providing better results when used in a classification task.

A Methodology to Measure the Diachronic Language Distance between Three Languages Based on Perplexity

“…Automatic periodization within a language is a related task, and for this aim, Degaetano-Ortlieb and Teich (2018) use relative entropy. For a similar aim, combination of perplexity and Recurrent Neural Networks (RNN) has been used for identifying temporal trends in a corpus of medieval charters (Boldsen, Agirrezabal, and Paggio, 2019).…”

Time-Aware Language Modeling for Historical Text Dating

“…Early studies employ features by manual work to recognize temporal expressions within documents (Dalli, 2006;Kanhabua and Nørvåg, 2016;Niculae et al, 2014), which suffer from the problem of generalization and coverage rate. Traditional machine learning methods focus on statistical features and learning models, such as Naïve Bayes (Boldsen and Wahlberg, 2021), SVM (Garcia-Fernandez et al, 2011) and Random Forests (Ciobanu et al, 2013). Recent studies turn to deep learning methods, and the experiments show their superior performances compared to traditional machine learning ones (Kulkarni et al, 2018;Liebeskind and Liebeskind, 2020;Yu and Huangfu, 2019;Ren et al, 2022).…”

“…torical documents for the construction of digital libraries (Baledent et al, 2020). Such task is also called historical text dating (Boldsen and Wahlberg, 2021), diachronic text evaluation (Popescu and Strapparava, 2015), or period classification (Tian and Kübler, 2021). Compared to other dating tasks, historical text dating is more challenging as explicit temporal mentions(e.g., time expressions) that help to determine the written date of a document usually do not appear in it (Toner and Han, 2019).…”

“…Compared to other dating tasks, historical text dating is more challenging as explicit temporal mentions(e.g., time expressions) that help to determine the written date of a document usually do not appear in it (Toner and Han, 2019). To solve it, current research on historical text dating focuses on document modeling, trying to find the relationship between time and linguistic features (Boldsen and Wahlberg, 2021).…”