Representation Learning for Type-Driven Composition

Wijnholds, Gijs; Sadrzadeh, Mehrnoosh; Clark, Stephen

doi:10.18653/v1/2020.conll-1.24

Cited by 13 publications

(36 citation statements)

References 26 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, the score achieved by SBERT is, by far, the highest correlation value reported on this dataset. Our results coincide exactly with those reported in [36] for this system and this dataset. It should be noted, however, that SBERT is a supervised approach as it relies on more than 1 million annotated sentence pairs.…”

Section: Discussionsupporting

confidence: 93%

“…In English, the best configuration is also achieved with a right-to-left strategy, 53, just considering the contextualized head but using lexico-syntactic units instead of lemmas. To the best of our knowledge, this value is very close to the highest score, 54, obtained by a compositional system on the English dataset [36], and outperforms other compositional methods whose values for the English dataset are also shown in Table 1 (last rows in left side), namely [37,38], and the neural network method reported in [39].…”

Section: Resultssupporting

confidence: 72%

See 1 more Smart Citation

Compositional Distributional Semantics with Syntactic Dependencies and Selectional Preferences

Gamallo¹

2021

Applied Sciences

View full text Add to dashboard Cite

This article describes a compositional model based on syntactic dependencies which has been designed to build contextualized word vectors, by following linguistic principles related to the concept of selectional preferences. The compositional strategy proposed in the current work has been evaluated on a syntactically controlled and multilingual dataset, and compared with Transformer BERT-like models, such as Sentence BERT, the state-of-the-art in sentence similarity. For this purpose, we created two new test datasets for Portuguese and Spanish on the basis of that defined for the English language, containing expressions with noun-verb-noun transitive constructions. The results we have obtained show that the linguistic-based compositional approach turns out to be competitive with Transformer models.

show abstract

Section: Discussionsupporting

confidence: 93%

Section: Resultssupporting

confidence: 72%

Compositional Distributional Semantics with Syntactic Dependencies and Selectional Preferences

Gamallo¹

2021

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…In the present paper we analyse the Gaussianity of the matrices in [20] using the permutation invariant matrix model of [8]. We again find strong evidence for approximate Gaussianity.…”

Section: Introductionmentioning

confidence: 62%

“…The construction of matrices in [1] was based on first constructing vectors for nouns and noun phrases, followed by using a linear regression method to construct the matrices as in earlier literature [13]. In the recent paper [20] the machine learning algorithm of [7] was extended to construct matrices for verbs.…”

Section: Introductionmentioning

confidence: 99%

Permutation invariant matrix statistics and computational language tasks

Huber¹,

Correia²,

Ramgoolam³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The Linguistic Matrix Theory programme introduced by Kartsaklis, Ramgoolam and Sadrzadeh is an approach to the statistics of matrices that are generated in type-driven distributional semantics, based on permutation invariant polynomial functions which are regarded as the key observables encoding the significant statistics. In this paper we generalize the previous results on the approximate Gaussianity of matrix distributions arising from compositional distributional semantics. We also introduce a geometry of observable vectors for words, defined by exploiting the graph-theoretic basis for the permutation invariants and the statistical characteristics of the ensemble of matrices associated with the words. We describe successful applications of this unified framework to a number of tasks in computational linguistics, associated with the distinctions between synonyms, antonyms, hypernyms and hyponyms.

show abstract

“…To interpret text fragments taking into account their grammatical features while staying in the vector framework, the dimension of the representation quickly scales up with the complexity of the syntactic type, which has been a limiting feature in distributional semantics implementations [32]. This motivates a representation of words as quantum states, counting on the potential of quantum computers to outperform the limitations of classical computation both in terms of memory use [33] and in processing efficiency [34].…”

Section: A Quantum States As Inputs Of a Quantum Circuitmentioning

confidence: 99%

Grover's Algorithm for Question Answering

Correia

Moortgat²,

Stoof³

2021

Preprint

View full text Add to dashboard Cite

Grover's algorithm, a well-know quantum search algorithm, allows one to find the correct item in a database, with quadratic speedup. In this paper we adapt Grover's algorithm to the problem of finding a correct answer to a natural language question in English, thus contributing to the growing field of Quantum Natural Language Processing. Using a grammar that can be interpreted as tensor contractions, each word is represented as a quantum state that serves as input to the quantum circuit. We here introduce a quantum measurement to contract the representations of words, resulting in the representation of larger text fragments. Using this framework, a representation for the question is found that contains all the possible answers in equal quantum superposition, and allows for the building of an oracle that can detect a correct answer, being agnostic to the specific question. Furthermore, we show that our construction can deal with certain types of ambiguous phrases by keeping the various different meanings in quantum superposition.

show abstract

Representation Learning for Type-Driven Composition

Cited by 13 publications

References 26 publications

Compositional Distributional Semantics with Syntactic Dependencies and Selectional Preferences

Compositional Distributional Semantics with Syntactic Dependencies and Selectional Preferences

Permutation invariant matrix statistics and computational language tasks

Grover's Algorithm for Question Answering

Contact Info

Product

Resources

About