Entity Disambiguation by Knowledge and Text Jointly Embedding

Abstract

For most entity disambiguation systems, the secret recipes are feature representations for mentions and entities, most of which are based on Bag-of-Words (BoW) representations. Commonly, BoW has several drawbacks: (1) It ignores the intrinsic meaning of words/entities; (2) It often results in high-dimension vector spaces and expensive computation; (3) For different applications, methods of designing handcrafted representations may be quite different, lacking of a general guideline. In this paper, we propose a different approach named EDKate. We first learn low-dimensional continuous vector representations for entities and words by jointly embedding knowledge base and text in the same vector space. Then we utilize these embeddings to design simple but effective features and build a two-layer disambiguation model. Extensive experiments on real-world data sets show that (1) The embedding-based features are very effective. Even a single one embedding-based feature can beat the combination of several BoW-based features. (2) The superiority is even more promising in a difficult set where the mention-entity prior cannot work well. (3) The proposed embedding method is much better than trivial implementations of some off-the-shelf embedding algorithms. (4) We compared our EDKate with existing methods/systems and the results are also positive.

Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

“…Such embedding models enable us to design NED models that capture the contextual information required to address NED. These models are typically based on conventional word embedding models (e.g., skip-gram (Mikolov et al, 2013)) that assign a fixed embedding to each word and entity (Yamada et al, 2016;Fang et al, 2016;Tsai and Roth, 2016;Cao et al, 2017;Ganea and Hofmann, 2017). In this study, we aim to test the effectiveness of the pretrained contextualized embeddings for NED.…”

Beyond word embeddings: learning entity and concept representations from large scale knowledge bases

“…Although the learning of the embeddings might seem straightforward, as it uses the standard skip-gram model, we see this as an advantage. On one hand, it allows our training to scale efficiently to huge vocabulary of words and concepts without the need for a lot of preprocessing (e.g., removing low frequent words and phrases as in Wang et al (2014); Fang et al (2016)). On the other hand, to learn from the knowledge graph contexts, we propose simple adaption to the skip-gram model (cf.…”

“…This is a simpler and more computationally efficient function than the scoring function proposed by previous approaches which learn from knowledge graphs (cf. Fang et al (2016)'s equation 1).…”

Multi-level Representations for Fine-Grained Typing of Knowledge Base Entities

“…We represent entities on three levels: entity, word and character. Our entity-level representation is similar to work on relation extraction (Wang et al, 2014;, entity linking (Yamada et al, 2016;Fang et al, 2016), and entity typing (Yaghoobzadeh and Schütze, 2015). Our word-level representation with distributional word embeddings is similarly used to represent entities for entity linking and relation extraction (Socher et al, 2013;Wang et al, 2014).…”