Tracing semantic change with Latent Semantic Analysis

Abstract

Research in historical semantics relies on the examination, selection, and interpretation of texts from corpora. Changes in meaning are tracked through the collection and careful inspection of examples that span decades and centuries. This process is inextricably tied to the researcher"s expertise and familiarity with the corpus. Consequently, the results tend to be difficult to quantify and put on an objective footing, and "big-picture" changes in the vocabulary other than the specific ones under investigation may be hard to keep track of. In this paper we present a method that uses Latent Semantic Analysis (Landauer, Foltz & Laham, 1998) to automatically track and identify semantic changes across a corpus. This method can take the entire corpus into account when tracing changes in the use of words and phrases, thus potentially allowing researchers to observe the larger context in which these changes occurred, while at the same time considerably reducing the amount of work required. Moreover, because this measure relies on readily observable co-occurrence data, it affords the study of semantic change a measure of objectivity that was previously difficult to attain. In this paper we describe our method and demonstrate its potential by applying it to several well-known examples of semantic change in the history of the English language.

Taming big data: Applying the experimental method to naturalistic data sets

“…Although do started out as a verb with a meaning that was quite relational, its meaning was still less context-sensitive in Middle English than it is in Modern English. By measuring the variety of contexts in which do appears, Sagi, Kaufmann, and Clark (2012) demonstrate this shift using corpus statistics, with results that correspond to Ellegård's hand-coded measures. The measure they use is essentially a measure of the variability of contexts in which the word appears within each period.…”

Short-Term Meaning Shift: A Distributional Exploration

“…Therefore, all models (including ours) spot semantic shift as a change in the word representation in different time periods. Among the most widely used techniques are Latent Semantic Analysis (Sagi et al, 2011;Jatowt and Duh, 2014), Topic Modeling (Wijaya and Yeniterzi, 2011), classic distributional representations based on co-occurence matrices of target words and context terms (Gulordava and Baroni, 2011). More recently, researchers have used word embeddings computed using the skip-gram model by Mikolov et al (2013).…”

Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change

“…c, Awful underwent a process of pejoration, as it shifted from meaning "full of awe" to meaning "terrible or appalling" (Simpson et al, 1989). are then compared across time-periods. This new direction has been effectively demonstrated in a number of case-studies (Sagi et al, 2011;Wijaya and Yeniterzi, 2011;Gulordava and Baroni, 2011;Jatowt and Duh, 2014) and used to perform largescale linguistic change-point detection (Kulkarni et al, 2014) as well as to test a few specific hypotheses, such as whether English synonyms tend to change meaning in similar ways (Xu and Kemp, 2015). However, these works employ widely different embedding approaches and test their approaches only on English.…”