Corpus-based Learning of Analogies and Semantic Relations

Abstract

Abstract. We present an algorithm for learning from unlabeled text, based on the Vector Space Model (VSM) of information retrieval, that can solve verbal analogy questions of the kind found in the SAT college entrance exam. A verbal analogy has the form A:B::C:D, meaning "A is to B as C is to D"; for example, mason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B, and the problem is to select the most analogous word pair, C:D, from a set of five choices. The VSM algorithm correctly answers 47% of a collection of 374 college-level analogy questions (random guessing would yield 20% correct; the average college-bound senior high school student answers about 57% correctly). We motivate this research by applying it to a difficult problem in natural language processing, determining semantic relations in noun-modifier pairs. The problem is to classify a noun-modifier pair, such as "laser printer", according to the semantic relation between the noun (printer) and the modifier (laser). We use a supervised nearest-neighbour algorithm that assigns a class to a given noun-modifier pair by finding the most analogous noun-modifier pair in the training data. With 30 classes of semantic relations, on a collection of 600 labeled noun-modifier pairs, the learning algorithm attains an F value of 26.5% (random guessing: 3.3%). With 5 classes of semantic relations, the F value is 43.2% (random: 20%). The performance is state-of-the-art for both verbal analogies and noun-modifier relations.

A uniform approach to analogies, synonyms, antonyms, and associations

“…Hofstadter (2007) claimed, "all meaning comes from analogies." In NLP, analogical algorithms have been applied to machine translation (Lepage and Denoual, 2005), morphology (Lepage, 1998), and semantic relations (Turney and Littman, 2005). Analogy provides a framework that has the potential to unify the field of semantics.…”

ALT-C 2012 Conference Proceedings - A confrontation with reality

“…In order to evaluate the proposed approach for analogy generation, we follow the method explained by Turney and Littman (Turney and Littman 2005) for evaluating analogies using a large corpus. In their study, Turney and Littman reported that their method can solve about 47% of multiple-choice analogy questions (compared to an average of 57% correct answers solved by high school students).…”

A Supervised Classification Approach for Measuring Relational Similarity between Word Pairs

“…The best performing individual module was based on Vector Space Model (VSM). In the VSM approach to measuring relational similarity [7], first a vector is created for a word-pair (X, Y) by counting the frequencies of various lexical patterns containing X and Y. In their experiments they used 128 manually created patterns such as "X of Y", "Y of X", "X to Y" and "Y to X".…”

“…3.1 to extract lexical patterns for 374 SAT multiplechoice analogy questions. This dataset was first proposed by Turney and Littman [7] as a benchmark dataset to evaluate relational similarity measures. Generally, there are six word pairs in each question (i.e.…”

“…Therefore, a random guessing algorithm would obtain a score of 20. The score reported by average senior high-school student is about 57 [7]. We performed 5-fold cross validation on SAT questions to evaluate the performance of the proposed method using the multiplicative feature function and the RBF kernel, which we found to be the best combination in Table 2.…”