Embedding of semantic predications

Abstract

This paper concerns the generation of distributed vector representations of biomedical concepts from structured knowledge, in the form of subject-relation-object triplets known as semantic predications. Specifically, we evaluate the extent to which a representational approach we have developed for this purpose previously, known as Predication-based Semantic Indexing (PSI), might benefit from insights gleaned from neural-probabilistic language models, which have enjoyed a surge in popularity in recent years as a means to generate distributed vector representations of terms from free text. To do so, we develop a novel neural-probabilistic approach to encoding predications, called Embedding of Semantic Predications (ESP), by adapting aspects of the Skipgram with Negative Sampling (SGNS) algorithm to this purpose. We compare ESP and PSI across a number of tasks including recovery of encoded information, estimation of semantic similarity and relatedness, and identification of potentially therapeutic and harmful relationships using both analogical retrieval and supervised learning. We find advantages for ESP in some, but not all of these tasks, revealing the contexts in which the additional computational work of neural-probabilistic modeling is justified.

Rediscovering Don Swanson:The Past, Present and Future of Literature-based Discovery

“…In particular, one can pose the question “King :: Queen as Man :: X?” and solve for X by identifying the difference vector which includes Man and lies closest to (King – Queen). Trevor Cohen has extensively explored the use of an analogy model for literature based discovery based on vector proximity (e.g., Mower et al, 2016; Cohen & Widdows, 2009, 2017; Cohen et al, 2010). …”

Predicting Adverse Drug-Drug Interactions with Neural Embedding of Semantic Predications

“…In the current work we propose an alternative approach to DDI prediction which builds upon Zitnik et al's graph conceptualization of the DDI prediction problem, but uses Embedding of Semantic Predications (ESP) 16 , which can address each of these challenges, for representation learning. ESP is a method for generating vector embeddings for biomedical concepts, such as drugs and side effects, from concept-relationship-concept triples called predications, using a neural network.…”

“…The implementation of these operators depends on the vector symbolic architecture approach used. Here we use the Binary Spatter Code 16,17 , in which binding is the elementwise XOR. Because XOR is its own inverse, the release and binding operators are equivalent in this implementation.…”

“…The superposition operator, which allows multiple vectors to be "added" together, retains the elementwise most common element (1 or 0) across the superposed vectors, with ties broken at random. The update steps are also modulated by a linearly decreasing learning rate ( ) and the similarity between the desired and current vectors, measured as the non-negative normalized Hamming Distance (NNHD) 16 . Analogous update steps are completed for negative samples, which are created by substituting random objects for the objects in positive triples (using -NNHD rather than 1-NNHD).…”

“…The weight matrices are updated so as to move (warfarin) closer to (bleeding) ⊗ (aspirin); (aspirin) closer to (warfarin) ⊘ (bleeding); and (bleeding) closer to (warfarin) ⊘ (aspirin). We refer the interested reader to Cohen et al 16 for a detailed exposition of the algorithm and evaluations across a range of pairwise entity-level tasks.…”