Applying Multi-Sense Embeddings for German Verbs to Determine Semantic Relatedness and to Detect Non-Literal Language

Abstract

Up to date, the majority of computational models still determines the semantic relatedness between words (or larger linguistic units) on the type level. In this paper, we compare and extend multi-sense embeddings, in order to model and utilise word senses on the token level. We focus on the challenging class of complex verbs, and evaluate the model variants on various semantic tasks: semantic classification; predicting compositionality; and detecting non-literal language usage. While there is no overall best model, all models significantly outperform a word2vec single-sense skip baseline, thus demonstrating the need to distinguish between word senses in a distributional semantic model.

From Word To Sense Embeddings: A Survey on Vector Representations of Meaning

“…Different works have tried to introduce sense representations in the context of compositionality (Köper & im Walde, 2017;Kober, Weeds, Wilkie, Reffin, & Weir, 2017), with different degrees of success. The main idea is to select the intended sense of a word and only introduce that specific meaning into the composition, either through context-based sense induction (Thater, Fürstenau, & Pinkal, 2011), exemplar-based representation (Reddy, Klapaftis, McCarthy, & Manandhar, 2011), or with the help of external resources, such as WordNet (Gamallo & Pereira-Fariña, 2017).…”

Is More Always Better? Testing the Addition Bias for German Language Statistics

“…To estimate the emotional valence of the stimulus words, we use a list of automatically generated affective norms for German lemmas by Köper and Schulte im Walde (2016). These ratings "were obtained via a supervised learning algorithm that can automatically calculate a numerical rating of a word" (Köper & Schulte im Walde, 2016: 2595 and rely on distributional information and four human-generated datasets. We decided to use this automatically generated dataset and not one of the smaller human-generated ones because we need maximum coverage of the German words used in the present study.…”

“…The affective norms dataset (Köper & Schulte im Walde, 2016) does not cover the words wenige and weniger (there is also no information for wenig which could be a candidate for substitution). Therefore, we had to exclude pairs 4 and 6 from the dataset.…”

„Mann“ is to “Donna” as「国王」is to « Reine » Adapting the Analogy Task for Multilingual and Contextual Embeddings

“…Other independently constructed datasets do exist (e.g., Venekoski and Vankka, 2017;Svoboda and Brychcín, 2018)crucially, covering all languages of interest to this study: in Chinese (Jin and Wu, 2012;Chen et al, 2015;, Dutch (Garneau et al, 2021), English (Turney 2008Mikolov et al 2013b, a.o. ), French (Grave et al, 2018), German (Köper et al, 2015), Italian (Berardi et al, 2015), and Spanish (Cardellino, 2019). On the other hand, these resources were created by different research groups and may contain items that are not easily comparable or of lesser quality.…”