The role of syntactic dependencies in compositional distributional semantics

Abstract

This article provides a preliminary semantic framework for Dependency Grammar in which lexical words are semantically defined as contextual distributions (sets of contexts) while syntactic dependencies are compositional operations on word distributions. More precisely, any syntactic dependency uses the contextual distribution of the dependent word to restrict the distribution of the head, and makes use of the contextual distribution of the head to restrict that of the dependent word. The interpretation of composite expressions and sentences, which are analyzed as a tree of binary dependencies, is performed by restricting the contexts of words dependency by dependency in a left-to-right incremental way. Consequently, the meaning of the whole composite expression or sentence is not a single representation, but a list of contextualized senses, namely the restricted distributions of its constituent (lexical) words. We report the results of two large-scale corpus-based experiments on two different natural language processing applications: paraphrasing and compositional translation.

Unsupervised Compositional Translation of Multiword Expressions

“…We abandon the traditional choice of representing the meaning of a phrase or sentence as a single vector. In our approach, the meaning of a composite expression is represented by a contextualized vector for each constituent word rather than by a single vector standing for the entire expression (Erk and Padó, 2008;Weir et al, 2016;Gamallo, 2017). This is in accordance with the main postulates of Dependency Grammar which only defines linguistic categories for words and relations, but not for composite units such as phrases or sentences.…”

Contextualized Translations of Phrasal Verbs with Distributional Compositional Semantics and Monolingual Corpora

“…The proposed method for contextualized translation relies on two strategies: compositional distributional semantics and bilingual extraction from monolingual corpora. The syntax-based compositional distributional algorithm described in Section 3 was tested against several monolingual data sets (with intransitive and transitive constructions) and the results of these experiments were reported in Gamallo (2017c). The method to extract bilingual lexicons described in Section 4 participated in the SemEval 2017 Task 10, being the best system using monolingual corpora in the English-Spanish crosslingual sub-task (Gamallo 2017a).…”

“…However, we prefer keeping the term contextualization because the compositional strategy we use is the same as that required for generating contextualized senses in the same language. Preliminary ideas underlying this method have been reported in Gamallo (2017c).…”

Compositional Semantics using Feature-Based Models from WordNet

“…Erk and Padó (2008) propose a method in which the combination of two words, a and b, returns two vectors: a vector a' representing the sense of a given the selectional preferences imposed by b, and a vector b' standing for the sense of b given the (inverse) selectional preferences imposed by a. A similar strategy is reported in Gamallo (2017). Our approach is an attempt to join the main ideas of these syntax-based models (namely, second-order vectors, selectional preferences and two returning words per combi-nation) in order to apply them to WordNet-based word representations.…”