A Type-Driven Vector Semantics for Ellipsis with Anaphora Using Lambek Calculus with Limited Contraction

Abstract

We develop a vector space semantics for verb phrase ellipsis with anaphora using type-driven compositional distributional semantics based on the Lambek calculus with limited contraction (LCC) of Jäger [18]. Distributional semantics has a lot to say about the statistical colocation-based meanings of content words, but provides little guidance on how to treat function words. Formal semantics on the other hand, has powerful mechanisms for dealing with relative pronouns, coordinators, and the like. Type-driven compositional distributional semantics brings these two models together. We review previous compositional distributional models of relative pronouns, coordination and a restricted account of ellipsis in the DisCoCat framework of Coecke et al. [7,6]. We show how DisCoCat cannot deal with general forms of ellipsis, which rely on copying of information, and develop a novel way of connecting typelogical grammar to distributional semantics by assigning vector interpretable lambda terms to derivations of LCC in the style of Muskens & Sadrzadeh [40]. What follows is an account of (verb phrase) ellipsis in which word meanings can be copied: the meaning of a sentence is now a program with non-linear access to individual word embeddings. We present the theoretical setting, workout examples, and demonstrate our results on a toy distributional model motivated by data.

Pregroup Grammars, Their Syntax and Semantics

“…A preliminary study of vector space semantics for sentences with the discourse phenomena, e.g. VP-ellipsis with anaphora, has also been pursued in the work done by my PhD student Gijs Wijnholds [48,47] towards his dissertation. There is some other relevant work in this area, but not directly via pregroups or vectors, e.g.…”

The calculus of language: explicit representation of emergent linguistic structure through type-theoretical paradigms

“…Moreover, the recent success of DNN models has motivated several attempts to use types to provide explicit representations for both semantic and syntactic structures of language, based on current techniques, and of embeddings and vector representations in particular. However, in the vast majority of the cases, a separation persists between the construction of atomic (mostly semantic) types (Abzianidze 2016; Krishnamurthy, Dasigi, and Gardner 2017; Choi et al 2018; Raiman and Raiman 2018; Lin and Ji 2019; Chen, Chen, and Van Durme 2020) and the establishment of (mostly syntactic) dependencies among them (Clark et al 2016; Coecke 2019; Wijnholds and Sadrzadeh 2019; Coecke, Sadrzadeh, and Clark 2010).…”

Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality

“…Somewhat orthogonally, Type Logical Grammars (TLGs) form highly compositional models of language by accurately modelling grammar, however they lack distributionality, in that such models do not accurately describe the distributional semantics of a word, only its grammatical role. Applications of type-logics with limited contraction and permutation to phenomena that witness discontinuities such as unbounded dependencies is a line of research initiated in [10,47], with a later boost in [43,44,46], and also more recently in [63]. Distributional Compositional Categorical Semantics (DisCoCat) [15] and the models of Preller [50,51], combine these two approaches using category theoretic methods, originally developed to model Quantum protocols.…”