LX-LR4DistSemEval: a collection of language resources for the evaluation of distributional semantic models of Portuguese

Querido, Andreia; Carvalho, Rita de Cássia Moura; Rodrigues, João; García, Marcos; Silva, João; Correia, Catarina; Rendeiro, Nuno; Pereira, Ricardo; Campos, Marisa; Branco, António

doi:10.26334/2183-9077/rapln3ano2017a15

Cited by 6 publications

(1 citation statement)

References 13 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding evaluating Portuguese distributional models, we see different types of intrinsic tasks in Modern Portuguese. Querido et al (2017) translated a set of English intrinsic datasets including an analogy test, word similarity, and concept categorization into Portuguese. Wilkens et al (2016) used Portuguese BabelNet to create a TOEFL-like test set called BSG targeting semantic relations.…”

Section: Related Workmentioning

confidence: 99%

BAHP: Benchmark of Assessing Word Embeddings in Historical Portuguese

Tian¹,

Jarrett²,

Torres³

et al. 2021

Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities An

View full text Add to dashboard Cite

High quality distributional models can capture lexical and semantic relations between words. Hence, researchers design various intrinsic tasks to test whether such relations are captured. However, most of the intrinsic tasks are designed for modern languages, and there is a lack of evaluation methods for distributional models of historical corpora. In this paper, we conducted BAHP: a benchmark of assessing word embeddings in Historical Portuguese, which contains four types of tests: analogy, similarity, outlier detection, and coherence. We examined word2vec models generated from two historical Portuguese corpora in these four test sets. The results demonstrate that our test sets are capable of measuring the quality of vector space models and can provide a holistic view of the model's ability to capture syntactic and semantic information. Furthermore, the methodology for the creation of our test sets can be easily extended to other historical languages.

show abstract

Section: Related Workmentioning

confidence: 99%