Learning Compositionality Functions on Word Embeddings for Modelling Attribute Meaning in Adjective-Noun Phrases

Abstract

Word embeddings have been shown to be highly effective in a variety of lexical semantic tasks. They tend to capture meaningful relational similarities between individual words, at the expense of lacking the capabilty of making the underlying semantic relation explicit. In this paper, we investigate the attribute relation that often holds between the constituents of adjective-noun phrases. We use CBOW word embeddings to represent word meaning and learn a compositionality function that combines the individual constituents into a phrase representation, thus capturing the compositional attribute meaning. The resulting embedding model, while being fully interpretable, outperforms countbased distributional vector space models that are tailored to attribute meaning in the two tasks of attribute selection and phrase similarity prediction. Moreover, as the model captures a generalized layer of attribute meaning, it bears the potential to be used for predictions over various attribute inventories without re-training.

Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition

“…In recent years, word embeddings have been used to predict the compositionality of phrases (Salehi et al, 2015;Cordeiro et al, 2016), and to identify the implicit relation in adjective-noun compositions (Hartung et al, 2017) and in noun compounds (Surtani and Paul, 2015;Dima, 2016;Shwartz and Waterson, 2018;Shwartz and Dagan, 2018).…”

Composing Noun Phrase Vector Representations

“…Although this asymmetry has been observed for nominal phrases as well, e.g. by Hartung et al (2017) who find that adjective representations capture more of the compositional semantics of an adjective-noun phrase than nouns do and implicitly also by Mitchell and Lapata (2010), whose composition functions give more weight to the adjectives than to the nouns, to our knowledge this is the first work that actively proposes and integrates this constraint into the composition process.…”

“…Early work on representing word sequences focused on bigram compositionality and considered various simple functions, such as vector addition and averaging (Mitchell and Lapata, 2010;Blacoe and Lapata, 2012), while already Turney (2012) integrated features for more meaningful relations. This early work focused on the representation of specific syntactic constructions and specific number of words and continues to be an ongo-ing research topic: representations of verb phrases (Hashimoto and Tsuruoka, 2016), noun phrases (Baroni and Zamparelli, 2010;Boleda et al, 2013;Dima, 2016), a combination of the two (Zanzotto et al, 2010;Wieting et al, 2015), nounnoun compositionality (Reddy et al, 2011;Hermann et al, 2012;Cordeiro et al, 2018), noun phrases attribute meaning (Hartung et al, 2017;Shwartz and Waterson, 2018), etc. This strand of research covers a variety of approaches ranging from the simple vector arithmetics mentioned to vector-matrix composition operations (Zanzotto et al, 2010;Guevara, 2010;Baroni and Zamparelli, 2010;Boleda et al, 2013), to the functional application of word vectors (Coecke et al, 2010; to RNNs (Wieting et al, 2015) and other supervised (Hartung et al, 2017;Shwartz and Waterson, 2018) or unsupervised approaches (Hermann et al, 2012).…”

“…Baseline approaches We include baseline operations from the literature that were recently shown to outperform complex deep architectures (White et al, 2015;Wieting et al, 2016;Arora et al, 2017). We use weighted elementwise vector addition (1) and multiplication (2) (Mitchell and Lapata, 2010;Turney, 2012;White et al, 2015;Hartung et al, 2017;Arora et al, 2017) and weighted elementwise average (3) (Mikolov et al, 2013;Wieting et al, 2016). Since addition and multiplication have been shown to perform so strongly and since multiplicative models have the drawback that the presence of zeroes in either of the vectors leads to information essentially being lost, we follow Mitchell and Lapata (2010) and also include a fourth equation, combining the addition and multiplication operations (4).…”

Compositional Morpheme Embeddings with Affixes as Functions and Stems as Arguments

“…Broadly speaking, this work can be seen as an extension of Baroni et al (2014)'s compositional distributional semantic framework to the sub-word level. At a more narrow level, our work is reminiscent of Baroni and Zamparelli (2010), who model adjectives as matrices and nouns as vectors, and work like Hartung et al (2017), which seeks to learn composition functions in addition to vector representations.…”