On the Linearity of Semantic Change: Investigating Meaning Variation via Dynamic Graph Models

Abstract

We consider two graph models of semantic change. The first is a time-series model that relates embedding vectors from one time period to embedding vectors of previous time periods. In the second, we construct one graph for each word: nodes in this graph correspond to time points and edge weights to the similarity of the word's meaning across two time points. We apply our two models to corpora across three different languages. We find that semantic change is linear in two senses. Firstly, today's embedding vectors (= meaning) of words can be derived as linear combinations of embedding vectors of their neighbors in previous time periods. Secondly, self-similarity of words decays linearly in time. We consider both findings as new laws/hypotheses of semantic change.

Semantic Change and Emerging Tropes In a Large Corpus of New High German Poetry

“…Eger and Mehler (2016) find that semantic change is linear in two senses: semantic self-similarity of words tends to decrease linearly in time and word vectors at time t can be written as linear combinations of words vectors at time t − 1, which allows to forecast meaning change. Regarding methods, Xu and Kemp (2015) work with simple distributional count vectors, while Hamilton et al (2016) and Eger and Mehler (2016) use lowdimensional dense vector representations. Both works use different approaches to map independently induced word vectors (across time) in a common space: Hamilton et al (2016) learn to align word vectors using a projection matrix while Eger and Mehler (2016) induce second-order embeddings by computing the similarity of words, in each time slot, to a reference vocabulary.…”

“…Further, we track the self-similarity of words, both with a change point analysis and by evaluating 'total selfsimilarity' of words over time. The former helps us to reconstruct literary periods, while the latter provides us with further evidence for the law of linearity of semantic change (Eger and Mehler, 2016) using our new method.…”

Tracing armed conflicts with diachronic word embedding models

“…Hamilton et al (2016a) proposed an important distinction between cultural shifts and linguistic drifts. They showed that global embedding-based measures, like comparing the similarities of words to all other words in the lexicon in (Eger and Mehler, 2016), are sensitive to regular processes of linguistic drift, while local measures (comparing restricted lists of nearest associates) are a better fit for more irregular cultural shifts in word meaning. We here follow this latter path, because our downstream task (detecting armed conflicts dynamics from semantic representations of country names) certainly presupposes cultural shifts in the associations for these country names (not a real change of dictionary meaning).…”

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)

“…This includes mainly (i), semantic similarity models assuming one sense for each word and then measuring its spatial displacement by a similarity metric (such as cosine) in a semantic vector space (Gulordava and Baroni, 2011;Xu and Kemp, 2015;Eger and Mehler, 2016;Hellrich and Hahn, 2016;Hamilton et al, 2016a,b) and (ii), word sense induction models (WSI) inferring for each word a probability distribution over different word senses (or topics) in turn modeled as a distribution over words (Wang and Mccallum, 2006;Bamman and Crane, 2011;Wijaya and Yeniterzi, 2011;Lau et al, 2012;Mihalcea and Nastase, 2012;Frermann and Lapata, 2016).…”