UWB at SemEval-2020 Task 1: Lexical Semantic Change Detection

Abstract

In this paper, we describe our method for detection of lexical semantic change, i.e., word sense changes over time. We examine semantic differences between specific words in two corpora, chosen from different time periods, for English, German, Latin, and Swedish. Our method was created for the SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection. We ranked 1 st in Sub-task 1: binary change detection, and 4 th in Sub-task 2: ranked change detection. Our method is fully unsupervised and language independent. It consists of preparing a semantic vector space for each corpus, earlier and later; computing a linear transformation between earlier and later spaces, using Canonical Correlation Analysis and Orthogonal Transformation; and measuring the cosines between the transformed vector for the target word from the earlier corpus and the vector for the target word in the later corpus.

LSCDiscovery: A shared task on semantic change discovery and detection in Spanish

“…All words with values above the threshold were classified as change, and values below were classified as no change. This approach has shown high performance in several previous studies and shared tasks (Schlechtweg et al, 2019;Pömsl and Lyapin, 2020;Kaiser et al, 2020b;Pražák et al, 2020).…”

Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection

“…According to Hu et al (2019) these models can ideally capture complex characteristics of word use, and how they vary across linguistic contexts. The results of SemEval-2020 Task 1 , however, show that contrary to this, the token-based embedding models (Beck, 2020;Kutuzov and Giulianelli, 2020) are heavily outperformed by the type-based ones (Pražák et al, 2020;Asgari et al, 2020). The SGNS model was not only widely used, but also performed best among the participants in the task.…”

Explaining and Improving BERT Performance on Lexical Semantic Change Detection

“…This means that not every word occurrence is considered individually (token-based); instead, a general vector representation that summarizes every occurrence of a word (including polysemous words) is created. The results of SemEval-2020 Task 1 and DIACR-Ita (Basile et al, 2020; demonstrated that overall type-based approaches (Asgari et al, 2020;Kaiser et al, 2020;Pražák et al, 2020) achieved better results than token-based approaches (Beck, 2020;Kutuzov and Giulianelli, 2020;Laicher et al, 2020). This is surprising, however, for two main reasons: (i) contextualized token-based approaches have significantly outperformed static type-based approaches in several NLP tasks over the past years (Ethayarajh, 2019).…”