Frame Identification as Categorization: Exemplars vs Prototypes in Embeddingland

Sikos, Jennifer; Padó, Sebastian

doi:10.18653/v1/w19-0425

Cited by 11 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With only a single entity per category, the prototype and exemplar models are equivalent; with multiple entities per category the exemplar model performs consistently slightly worse than the NameBased prototype model, but with the same qualitative pattern (performance significantly above NounBased for

n \geq 4

). This is in line with studies which indicate that for current representation learning approaches, the nonlinear decision boundaries of exemplar models may not provide advantages over prototype models (Sikos & Padó, 2019).…”

Section: Methodssupporting

confidence: 90%

Distributional Models of Category Concepts Based on Names of Category Members

et al. 2021

Self Cite

View full text Add to dashboard Cite

Cognitive scientists have long used distributional semantic representations of categories. The predominant approach uses distributional representations of category‐denoting nouns, such as “city” for the category city. We propose a novel scheme that represents categories as prototypes over representations of names of its members, such as “Barcelona,” “Mumbai,” and “Wuhan” for the category city. This name‐based representation empirically outperforms the noun‐based representation on two experiments (modeling human judgments of category relatedness and predicting category membership) with particular improvements for ambiguous nouns. We discuss the model complexity of both classes of models and argue that the name‐based model has superior explanatory potential with regard to concept acquisition.

show abstract

n \geq 4

Section: Methodssupporting

confidence: 90%

Distributional Models of Category Concepts Based on Names of Category Members

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The latest generation of embedding architectures are the so-called transformers which can learn contextual dependencies in an unsupervised fashion and construct context-dependent meaning representations: tree will receive one embedding in the phrase the tree in the forest and another one in the phrase dependency tree. Not surprisingly, one of the best-known transformer models, BERT (Devlin et al, 2019), is the basis of state-of-the-art frame identification models for English (Sikos andPadó, 2019, Tan andNa, 2019).…”

Section: Modeling Multilingual Frame Identificationmentioning

confidence: 99%

Improving Multilingual Frame Identification by Estimating Frame Transferability

Sikos

Roth

Padó

2022

LiLT

Self Cite

View full text Add to dashboard Cite

A recent research direction in computational linguistics involves efforts to make the field, which used to focus primarily on English, more multilingual and inclusive. However, resource creation often remains a bottleneck for many languages, in particular at the semantic level. In this article, we consider the case of frame-semantic annotation. We investigate how to perform frame selection for annotation in a target language by taking advantage of existing annotations in different, supplementary languages, with the goal of reducing the required annotation effort in the target language. We measure success by training and testing frame identification models for the target language. We base our selection methods on measuring frame transferability in the supplementary language, where we estimate which frames will transfer poorly, and therefore should receive more annotation, in the target language. We apply our approach to English, German, and French – three languages which have annotations that are similar in size as well as frames with overlapping lexicographic definitions. We find that transferability is indeed a useful indicator and supports a setup where a limited amount of target language data is sufficient to train frame identification systems.

show abstract

“…Thus, the embeddings produced by these new frameworks are said to be contextualized, as opposed to the static vectors produced by the earlier frameworks, and they aim at modeling the specific sense assumed by the word in context (Wiedemann et al 2019). Interestingly, the distinction between traditional and contextualized embeddings has been recently discussed by drawing a parallel between the prototype and exemplar models of categorization in cognitive psychology (Sikos and Padó 2019).…”

Section: From Static Distributional Models To Contextualized Embeddingsmentioning

confidence: 99%

Decoding Word Embeddings with Brain-Based Semantic Features

Chersoni

Santus²,

Huang

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

Word embeddings are vectorial semantic representations built with either counting or predicting techniques aimed at capturing shades of meaning from word co-occurrences. Since their introduction, these representations have been criticised for lacking interpretable dimensions. This property of word embeddings limits our understanding of the semantic features they actually encode. Moreover, it contributes to the “black box” nature of the tasks in which they are used, since the reasons of word embeddings performance often remain opaque to humans. In this contribution, we explore the semantic properties encoded in word embeddings by mapping them onto interpretable vectors, consisting of explicit and neurobiologically-motivated semantic features (Binder et al. 2016). Our exploration takes into account different types of embeddings, including factorized count vectors and predict models (e.g., Skip-Gram, GloVe, etc.), as well as the most recent contextualized representations (i.e., ELMo and BERT). In our analysis, we first evaluate the quality of the mapping in a retrieval task, then we shed lights on the semantic features that are better encoded in each embedding type. A large number of probing tasks is finally set to assess how the original and the mapped embeddings perform in discriminating semantic categories. For each probing task, we identify the most relevant semantic features and we show that there is a correlation between the embedding performance and how they encode those features. This study sets itself as a step forward in understanding which aspect of meaning are captured by vector spaces, by proposing a new and simple method to carve human-interpretable semantic representations from distributional vectors.

show abstract

Frame Identification as Categorization: Exemplars vs Prototypes in Embeddingland

Cited by 11 publications

References 30 publications

Distributional Models of Category Concepts Based on Names of Category Members

Distributional Models of Category Concepts Based on Names of Category Members

Improving Multilingual Frame Identification by Estimating Frame Transferability

Decoding Word Embeddings with Brain-Based Semantic Features

Contact Info

Product

Resources

About