Learning distributed event representations with a multi-task approach

Hong, Xudong; Sayeed, Asad; Demberg, Vera

doi:10.18653/v1/s18-2002

Cited by 8 publications

(9 citation statements)

References 32 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several models have tried to tackle the "dynamic" version of the thematic fit task, either based on classical distributional spaces (Chersoni et al, 2016(Chersoni et al, , 2019 or on more sophisticated neural network architectures (Tilk et al, 2016;Hong et al, 2018). On the evaluation side, those works made use of the experimental materials of the study by Lenci (2011), which are, however, limited to agentverb-patient triples.…”

Section: Related Workmentioning

confidence: 99%

Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge

Pedinotti¹,

Rambelli²,

Chersoni³

et al. 2021

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

View full text Add to dashboard Cite

Prior research has explored the ability of computational models to predict a word semantic fit with a given predicate. While much work has been devoted to modeling the typicality relation between verbs and arguments in isolation, in this paper we take a broader perspective by assessing whether and to what extent computational approaches have access to the information about the typicality of entire events and situations described in language (Generalized Event Knowledge).Given the recent success of Transformers Language Models (TLMs), we decided to test them on a benchmark for the dynamic estimation of thematic fit. The evaluation of these models was performed in comparison with SDM, a framework specifically designed to integrate events in sentence meaning representations, and we conducted a detailed error analysis to investigate which factors affect their behavior. Our results show that TLMs can reach performances that are comparable to those achieved by SDM. However, additional analysis consistently suggests that TLMs do not capture important aspects of event knowledge, and their predictions often depend on surface linguistic features, such as frequent words, collocations and syntactic patterns, thereby showing sub-optimal generalization abilities.

show abstract

Section: Related Workmentioning

confidence: 99%

Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge

Pedinotti¹,

Rambelli²,

Chersoni³

et al. 2021

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

View full text Add to dashboard Cite

show abstract

“…Thematic fit is a psycholinguistic notion similar to selectional preferences, the main difference being that the latter involve the satisfaction of constraints on discrete semantic features of the arguments, while thematic fit is a continuous value expressing the degree of compatibility between an argument and a semantic role (McRae et al 1998). Distributional models for thematic fit estimation have been proposed by several authors (Erk 2007; Baroni and Lenci 2010; Erk et al 2010; Lenci 2011; Sayeed et al 2015; Greenberg et al 2015; Santus et al 2017; Tilk et al 2016; Hong et al 2018). While thematic fit data sets typically include human-elicited typicality scores for argument–filler pairs taken in isolation, DTFit includes tuples of arguments of different length, so that the typicality value of an argument depends on its interaction with the other arguments in the tuple.…”

Section: Methodsmentioning

confidence: 99%

“… k It should also be noticed that our LSTM baseline has been trained on simple syntactic dependencies, while state-of-the-art neural models rely simultaneously on dependencies and semantic role labels (Tilk et al 2016; Hong et al 2018). …”

mentioning

confidence: 99%

A structured distributional model of sentence meaning and processing

Chersoni

Santus²,

Pannitto

et al. 2019

Nat. Lang. Eng.

View full text Add to dashboard Cite

Most compositional distributional semantic models represent sentence meaning with a single vector. In this paper, we propose a structured distributional model (SDM) that combines word embeddings with formal semantics and is based on the assumption that sentences represent events and situations. The semantic representation of a sentence is a formal structure derived from discourse representation theory and containing distributional vectors. This structure is dynamically and incrementally built by integrating knowledge about events and their typical participants, as they are activated by lexical items. Event knowledge is modelled as a graph extracted from parsed corpora and encoding roles and relationships between participants that are represented as distributional vectors. SDM is grounded on extensive psycholinguistic research showing that generalized knowledge about events stored in semantic memory plays a key role in sentence comprehension.We evaluate SDMon two recently introduced compositionality data sets, and our results show that combining a simple compositionalmodel with event knowledge constantly improves performances, even with dif ferent types of word embeddings.

show abstract

“…Our approach captures a rich variety of explicit semantic connections among complex events. (Hong et al, 2018) learns distributed event representations using supervised multi-task learning, while our framework is based on unsupervised learning. Network Embedding.…”

Section: Related Workmentioning

confidence: 99%

GENE: Global Event Network Embedding

Zeng

Lai³

et al. 2021

Proceedings of the Fifteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-15)

View full text Add to dashboard Cite

Current methods for event representation ignore related events in a corpus-level global context. For a deep and comprehensive understanding of complex events, we introduce a new task, Event Network Embedding, which aims to represent events by capturing the connections among events. We propose a novel framework, Global Event Network Embedding (GENE), that encodes the event network with a multi-view graph encoder while preserving the graph topology and node semantics. The graph encoder is trained by minimizing both structural and semantic losses. We develop a new series of structured probing tasks, and show that our approach effectively outperforms baseline models on node typing, argument role classification, and event coreference resolution. 1

show abstract

Learning distributed event representations with a multi-task approach

Cited by 8 publications

References 32 publications

Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge

Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge

A structured distributional model of sentence meaning and processing

GENE: Global Event Network Embedding

Contact Info

Product

Resources

About