Semeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings

Abstract

Word embeddings have advanced the state of the art in NLP across numerous tasks. Understanding the contents of dense neural representations is of utmost interest to the computational semantics community. We propose to focus on relating these opaque word vectors with human-readable definitions, as found in dictionaries. This problem naturally divides into two subtasks: converting definitions into embeddings, and converting embeddings into definitions. This task was conducted in a multilingual setting, using comparable sets of embeddings trained homogeneously.

1Cademy at Semeval-2022 Task 1: Investigating the Effectiveness of Multilingual, Multitask, and Language-Agnostic Tricks for the Reverse Dictionary Task

“…Reverse dictionary results are evaluated using three metrics: mean squared error (MSE) between the reconstructed and reference embeddings, cosine similarity (COS) between the reconstructed embedding and the reference embedding, and the cosinebased ranking (RANK) between the reconstructed and reference embeddings, measuring the number of other test items having higher cosine with the reconstructed embedding than with the reference embedding (Mickus et al, 2022).…”

“…Our paper is devoted to the performance comparison of different neural network structures, multilingual and multitask tricks, and elaborating on language-agnostic or bidirectional structure helpfulness. The competition (Mickus et al, 2022) has significant potential in contributing pretraining process acceleration, low-resource language model development, and commonsense using. Furthermore, the task is of high importance for explainable AI and natural language processing since it models direct mapping from human-readable data to machine-readable data.…”

LingJing at SemEval-2022 Task 1: Multi-task Self-supervised Pre-training for Multilingual Reverse Dictionary

“…In this task, the performance of the system is evaluated through three evaluation indicators (Mickus et al, 2022). Mean squared error (MSE) between the submission's reconstructed embedding and the reference embedding.…”

MMG at SemEval-2022 Task 1: A Reverse Dictionary approach based on a review of the dataset from a lexicographic perspective

“…The CODWOE (Comparison of Word Glosses and Word Embeddings) task at SemEval-2022 (Mickus et al, 2022) encouraged participants to analyze the relation between two types of semantic descriptions, word embeddings and dictionary glosses, by proposing two sub-tasks: Reverse Dictionary (RD) (Hill et al, 2016), in which participants must generate vectors from glosses, and Definition Modeling (DM) (Noraset et al, 2017), in which participants must generate glosses from vectors. These subtasks aim to be useful for explainable Artificial Intelligence (AI) by including human-readable and machine-readable data.…”