New experiments in distributional representations of synonymy

Abstract

Recent work on the problem of detecting synonymy through corpus analysis has used the Test of English as a Foreign Language (TOEFL) as a benchmark. However, this test involves as few as 80 questions, prompting questions regarding the statistical significance of reported results. We overcome this limitation by generating a TOEFL-like test using WordNet, containing thousands of questions and composed only of words occurring with sufficient corpus frequency to support sound distributional comparisons. Experiments with this test lead us to a similarity measure which significantly outperforms the best proposed to date. Analysis suggests that a strength of this measure is its relative robustness against polysemy.

Specializing Word Embeddings for Similarity or Relatedness

“…In the well-known TOEFL evaluation (Freitag et al, 2005) models are required to identify true synonyms to question words from a selection of possible answers. To test our models on this task, for each question in the dataset, we rank the multiple-choice answers according to the cosine similarity between their word embeddings and that of the target word, and choose the highestranked option.…”

A Survey on Portuguese Lexical Knowledge Bases: Contents, Comparison and Combination

“…The B 2 SG [25] test is similar to the WordNet-Based Synonymy Test [26], but based on the Portuguese part of BabelNet [13] and partially evaluated by humans. It contains frequent Portuguese nouns and verbs (target), each followed by four candidates, from which only one is related, and is organised in six files: two for synonymy, two for hypernymy, and two for antonymy, respectively, between nouns and for verbs.…”

Comparing Distributional and Mirror Translation Similarities for Extracting Synonyms

“…As a secondary evaluation, authors often use TOEFL synonymy tests [19,6] where the task is to distinguish the synonym of a given word in a given context, among four candidate items. This is a sort of easier word disambiguation test where the task is to separate a word from unrelated distractors, instead of distinguishing between word senses.…”

“…We lemmatized both French and English sentences using TreeTager. 6 Then, we trained in both directions 7 (English-to-French and French-to-English) statistical translation models, running the Giza++ toolkit in its standard setting. 8 Our translation mirror approach makes use of the lexical distribution of the two models, 9 p e2f and p f 2e , we obtained this way (see tab 3 for an example).…”

“…Also popular was distributional analysis, comparing words via their common contexts of use, or syntactic dependencies [4,5] in order to define another kind of semantic similarity. These approaches went on using more readily available resources in more languages [6]. More recently, a similar approach has gained popularity using bitexts in parallell corpora.…”