Towards Holistic Concept Representations: Embedding Relational Knowledge, Visual Attributes, and Distributional Word Semantics

Thoma, Steffen; Rettinger, Achim; Both, Fabian

doi:10.1007/978-3-319-68288-4_41

Cited by 26 publications

(19 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A high purity means that resources with similar type vectors (e.g., presidents who are also authors) are located close to each other in the embedding space, which is a wanted characteristic of a KGE. In our second evaluation, we performed a type prediction experiment in a manner akin to [10,17]. For each resource x ∈ S, we used the µ closest embeddings of x to predict x's type vector.…”

Section: Time Complexitymentioning

confidence: 99%

A Physical Embedding Model for Knowledge Graphs

Demir

Ngomo

2020

Semantic Technology

View full text Add to dashboard Cite

Knowledge graph embedding methods learn continuous vector representations for entities in knowledge graphs and have been used successfully in a large number of applications. We present a novel and scalable paradigm for the computation of knowledge graph embeddings, which we dub PYKE. Our approach combines a physical model based on Hooke's law and its inverse with ideas from simulated annealing to compute embeddings for knowledge graphs efficiently. We prove that PYKE achieves a linear space complexity. While the time complexity for the initialization of our approach is quadratic, the time complexity of each of its iterations is linear in the size of the input knowledge graph. Hence, PYKE's overall runtime is close to linear. Consequently, our approach easily scales up to knowledge graphs containing millions of triples. We evaluate our approach against six state-of-the-art embedding approaches on the Drug-Bank and DBpedia datasets in two series of experiments. The first series shows that the cluster purity achieved by PYKE is up to 26% (absolute) better than that of the state of art. In addition, PYKE is more than 22 times faster than existing embedding solutions in the best case. The results of our second series of experiments show that PYKE is up to 23% (absolute) better than the state of art on the task of type prediction while maintaining its superior scalability. Our implementation and results are open-source and are available at http: //github.com/dice-group/PYKE.

show abstract

Section: Time Complexitymentioning

confidence: 99%

A Physical Embedding Model for Knowledge Graphs

Demir

Ngomo

2020

Semantic Technology

View full text Add to dashboard Cite

show abstract

“…also fused text-based representations with imagebased representations (Bruni et al, 2014;Lazaridou et al, 2015;Chrupala et al, 2015;Mao et al, 2016;Silberer et al, 2017;Collell et al, 2017;Zablocki et al, 2018) and representations derived from a knowledge-graph (Thoma et al, 2017). More recently, gating-based approaches have been developed for fusing traditional word embeddings with visual representations.…”

Section: Related Workmentioning

confidence: 99%

Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search

Kiros¹,

Chan²,

Hinton³

2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

We introduce Picturebook, a large-scale lookup operation to ground language via 'snapshots' of our physical world accessed through image search. For each word in a vocabulary, we extract the top-k images from Google image search and feed the images through a convolutional network to extract a word embedding. We introduce a multimodal gating function to fuse our Picturebook embeddings with other word representations. We also introduce Inverse Picturebook, a mechanism to map a Picturebook embedding back into words. We experiment and report results across a wide range of tasks: word similarity, natural language inference, semantic relatedness, sentiment/topic classification, image-sentence ranking and machine translation. We also show that gate activations corresponding to Picturebook embeddings are highly correlated to human judgments of concreteness ratings.

show abstract

“…There are notable examples showcasing the influence of neural approaches to knowledge acquisition and representation learning on the broad area of Semantic Web technologies. These include, among oth-ers, ontology learning [40,49,65], learning structured query languages from natural language [69], ontology alignment [20,28,35,52], ontology annotation [15,58], joined relational and multi-modal knowledge representations [62], and relation prediction [1,59]. Ontologies, on the other hand, have been repeatedly utilized as background knowledge for machine learning tasks.…”

Section: Introductionmentioning

confidence: 99%

“…As an example, there is a myriad of hybrid approaches for learning linguistic representations by jointly incorporating corpus-based evidence and semantic resources [13,25,27,33,50]. This interplay between structured knowledge and corpus-based approaches has given way to knowledge graph embeddings, which in turn have proven useful for tasks such as hypernym discovery [21], collocation discovery and classification [22], word sense disambiguation [12,54], joined relational and multi-modal knowledge representations [62] and many others.…”

Section: Introductionmentioning

confidence: 99%

Special issue on Semantic Deep Learning

Gromann

Espinosa-Anke

Declerck

2019

View full text Add to dashboard Cite

Numerous success use cases involving deep learning have recently started to be propagated to the Semantic Web. Approaches range from utilizing structured knowledge in the training process of neural networks to enriching such architectures with ontological reasoning mechanisms. Bridging the neural-symbolic gap by joining deep learning and Semantic Web not only holds the potential of improving performance but also of opening up new avenues of research. This editorial introduces the Semantic Web Journal special issue on Semantic Deep Learning, which brings together Semantic Web and deep learning research. After a general introduction to the topic and a brief overview of recent contributions, we continue to introduce the submissions published in this special issue.

show abstract

Towards Holistic Concept Representations: Embedding Relational Knowledge, Visual Attributes, and Distributional Word Semantics

Cited by 26 publications

References 25 publications

A Physical Embedding Model for Knowledge Graphs

A Physical Embedding Model for Knowledge Graphs

Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search

Special issue on Semantic Deep Learning

Contact Info

Product

Resources

About