AMORE-UPF at SemEval-2018 Task 4: BiLSTM with Entity Library

Aina, Laura; Silberer, Carina; Sorodoc, Ionut-Teodor; Westera, Matthijs; Boleda, Gemma

doi:10.18653/v1/s18-1008

Cited by 4 publications

(17 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ENTLIB model ( Figure 3) is an adaptation of our previous work in Aina et al (2018), which was the winner of the SemEval 2018 Task 4 competition. This model adds a simple memory module that is expected to represent entities because its vectors are tied to the output classes (accordingly, Aina et al, 2018, call this module entity library). We call this memory 'static', since it is updated only during training, after which it remains fixed.…”

Section: Entlib (Static Memory)mentioning

confidence: 99%

“…Macro-average F 1 -score on all entities, the most stringent, was the criterion to define the leaderboard. Table 1 gives our results in the two evaluations, comparing the models described in Section 3 to the best performing models in the SemEval 2018 Task 4 competition (Aina et al, 2018;Park et al, Figure 5: Accuracy on entities with high (>1000), medium (20-1000), and low (<20) frequency. 2018).…”

Section: Character Identificationmentioning

confidence: 99%

“…http://friends.wikia.com.13 The referring expression is the whole DP, This person, but we follow the method inAina et al 2018 of asking for reference resolution at the head noun.14 Models require inputting a speaker; we use speaker UN-…”

mentioning

confidence: 99%

See 2 more Smart Citations

What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

Aina¹,

Silberer²,

Sorodoc³

et al. 2019

Proceedings of the 2019 Conference of the North

Self Cite

View full text Add to dashboard Cite

Humans use language to refer to entities in the external world. Motivated by this, in recent years several models that incorporate a bias towards learning entity representations have been proposed. Such entity-centric models have shown empirical success, but we still know little about why.In this paper we analyze the behavior of two recently proposed entity-centric models in a referential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4). We show that these models outperform the state of the art on this task, and that they do better on lower frequency entities than a counterpart model that is not entity-centric, with the same model size. We argue that making models entitycentric naturally fosters good architectural decisions. However, we also show that these models do not really build entity representations and that they make poor use of linguistic context. These negative results underscore the need for model analysis, to test whether the motivations for particular architectures are borne out in how models behave when deployed.1 Note the analogy with traditional models in formal linguistics like Discourse Representation Theory (Kamp and Reyle, 2013). 2 Source code for our model, the training procedure and the new dataset is published on https://github.com/ amore-upf/analysis-entity-centric-nns.3 https://competitions.codalab.org/ competitions/17310.

show abstract

Section: Entlib (Static Memory)mentioning

confidence: 99%

Section: Character Identificationmentioning

confidence: 99%

See 1 more Smart Citation

What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

Aina¹,

Silberer²,

Sorodoc³

et al. 2019

Proceedings of the 2019 Conference of the North

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the implementation of Aina et al (2018), which is related to our proposal, pronouns are learned from semantically annotated dialogue of character references (obtained from data of the TV series The authors assume that 'the LSTM can learn to simply forward the speaker embedding unchanged in the case of pronoun I' (Aina et al, 2018, p. 68). But simply mapping I to whatever is left to the colon still misses perspectivity, which is the crucial issue about first and second person pronouns, cf.…”

Section: Pronoun Gamesmentioning

confidence: 99%

“…Obviously, the bootstrapping problem completely undermines the DH's claim to provide a semantic theory. However, work that aims at accounting for such 'higher order' phenomena in distributional terms but drawing on additional resources or annotations-like Herbelot and Vecchi (2015), who map distributional information onto quantified sentences, Gupta et al (2015), who map distributions to knowledge base information, or Aina et al (2018), who employ distributional semantics on character annotations within a specifically designed network model-puts its emphasis not on distributional semantic representation, but on how to learn these representations. We observe a further, de facto construal of DH here, a construal that draws on the notion of supervised learning, triggered by the procedures of data extraction or machine learning.…”

Section: Introductionmentioning

confidence: 99%

Distribution is not enough: going Firther

Lücking¹,

Cooper²,

Larsson³

et al. 2019

Proceedings of the Sixth Workshop on Natural Language and Computer Science

View full text Add to dashboard Cite

Much work in contemporary computational semantics follows the distributional hypothesis (DH), which is understood as an approach to semantics according to which the meaning of a word is a function of its distribution over contexts which is represented as vectors (word embeddings) within a multi-dimensional semantic space. In practice, use is identified with occurrence in text corpora, though there are some efforts to use corpora containing multi-modal information. In this paper we argue that the distributional hypothesis is intrinsically misguided as a selfsupporting basis for semantics, as Firth was entirely aware. We mention philosophical arguments concerning the lack of normativity within DH data. Furthermore, we point out the shortcomings of DH as a model of learning, by discussing a variety of linguistic classes that cannot be learnt on a distributional basis, including indexicals, proper names, and wh-phrases. Instead of pursuing DH, we sketch an account of the problematic learning cases by integrating a rich, Firthian notion of dialogue context with interactive learning in signalling games backed by in probabilistic Type Theory with Records. We conclude that the success of the DH in computational semantics rests on a post hoc effect: DS presupposes a referential semantics on the basis of which utterances can be produced, comprehended and analysed in the first place.

show abstract

Challenges in including extra-linguistic context in pre-trained language models

Sorodoc¹,

Aina²,

Boleda³

2022

Proceedings of the Third Workshop on Insights From Negative Results in NLP

View full text Add to dashboard Cite

To successfully account for language, computational models need to take into account both the linguistic context (the content of the utterances) and the extra-linguistic context (for instance, the participants in a dialogue). We focus on a referential task that asks models to link entity mentions in a TV show to the corresponding characters, and design an architecture that attempts to account for both kinds of context. In particular, our architecture combines a previously proposed specialized module (an "entity library") for character representation with transfer learning from a pre-trained language model. We find that, although the model does improve linguistic contextualization, it fails to successfully integrate extra-linguistic information about the participants in the dialogue. Our work shows that it is very challenging to incorporate extra-linguistic information into pretrained language models.

show abstract

AMORE-UPF at SemEval-2018 Task 4: BiLSTM with Entity Library

Cited by 4 publications

References 11 publications

What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

Distribution is not enough: going Firther

Challenges in including extra-linguistic context in pre-trained language models

Contact Info

Product

Resources

About