Jointly Learning Word Representations and Composition Functions Using Predicate-Argument Structures

Abstract

We introduce a novel compositional language model that works on PredicateArgument Structures (PASs). Our model jointly learns word representations and their composition functions using bagof-words and dependency-based contexts. Unlike previous word-sequencebased models, our PAS-based model composes arguments into predicates by using the category information from the PAS. This enables our model to capture longrange dependencies between words and to better handle constructs such as verbobject and subject-verb-object relations. We verify this experimentally using two phrase similarity datasets and achieve results comparable to or higher than the previous best results. Our system achieves these results without the need for pretrained word vectors and using a much smaller training corpus; despite this, for the subject-verb-object dataset our model improves upon the state of the art by as much as ∼10% in relative performance.

Drug drug interaction extraction from biomedical literature using syntax convolutional neural network

“…Hashimoto et al (2014) learned the word embedding using predicate-argument structure contexts and used it to measure semantic similarity between short phrases. In their methods, the syntactic information is introduced by constructing the syntactic contexts instead of the normal linear contexts (i.e.…”

“…In contrast to previous syntactic context word embeddings (Hashimoto et al , 2014; Levy and Goldberg, 2014), our embedding is learned only based on the concise syntax word sequence, which represents a sentence’s syntactic structure while discards the less important words. It is simple but proven to be effective for DDI extraction by our experiments.…”

Task-Oriented Learning of Word Embeddings for Semantic Relation Classification

“…Hence, the word cause has two embeddings: one in N and another in W. In general cause is used as a noun and a verb, and thus we expect the noun embeddings to capture the meanings focusing on their noun usage. This is inspired by some recent work on word representations that explicitly assigns an independent representation for each word usage according to its partof-speech tag (Baroni and Zamparelli, 2010;Grefenstette and Sadrzadeh, 2011;Hashimoto et al, 2013;Hashimoto et al, 2014;Kartsaklis and Sadrzadeh, 2013).…”

Compositional Semantics using Feature-Based Models from WordNet

“…(Milajevs et al, 2014) 0.46 (Polajnar et al, 2015) 0.35 (Hashimoto et al, 2014) 0.48 (Hashimoto and Tsuruoka, 2015) 0.48 Human agreement 0.75 Table 4: Spearman correlation for transitive expressions using the benchmark by Grefenstette and Sadrzadeh (2011) Table 4 shows the Spearman's correlation values (ρ) obtained by all the different versions built from our model WN. The best score was achieved by averaging the head and dependent similarity values derived from the n-vn (right-to-left) strategy.…”