MMKRL: A robust embedding approach for multi-modal knowledge graph representation learning

Lu, Xinyu; Wang, Lifang; Jiang, Zejun; He, Shichang; Liu, Shizhong

doi:10.1007/s10489-021-02693-9

Cited by 15 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GAN proposes to train a pair of discriminator and generator by playing a min-max game between them and achieve better performance, which has been widely used in various fields such as computer vision [1,19], natural language processing [9,58], information retrieval [44,59], and recommender systems [50,52,58]. In the KGC field, there are some methods [4,30,39,46] employs a GAN-based framework to enhance the negative sampling [3] process. For example, KBGAN [4] utilizes reinforcement learning (RL) with GAN to learn a better sampling policy.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…Existing MMKGC methods [22,30,42,51] usually treat the multimodal information of entities as multi-modal embeddings and incorporate these multi-modal embeddings to enhance the entity representations. However, these methods neglect two vital problems for MMKGC in real scenarios, which can be concluded as the diversity problem and the imbalance problem.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

NativE: Multi-modal Knowledge Graph Completion in the Wild

Zhang,

Chen,

Guo

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Section: Generative Adversarial Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

NativE: Multi-modal Knowledge Graph Completion in the Wild

Zhang,

Chen,

Guo

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…These methods use information from other modalities as supplementary information for embedded representation acquisition. MMKRL [16] (multi-modal knowledge representation learning) also selects an adversarial training strategy to enhance the robustness, which is rarely considered in existing multi-modal knowledge representation learning methods.…”

Section: Related Work a Multi-modal Knowledge Representation Learningmentioning

confidence: 99%

Multi-Modal Entity Alignment Using Uncertainty Quantification for Modality Importance

Hama

Matsubara

2023

IEEE Access

View full text Add to dashboard Cite

The knowledge graphs are structured data utilized for information retrieval purposes. Entity alignment using multi-modal supplementary information plays an important role in knowledge graph integration. However, if the supplementary information is missing or incorrect, it can negatively impact the retrieval of information. If we can quantify the usefulness of the information for retrieval as a degree of importance, the influence of unimportant supplementary information can be reduced. In this study, we proposed a method that quantifies the importance of each piece of information by using a probability distribution. Our proposed method improves an existing method by 7.7% and 7.3% in H@1 on two datasets (FB15K-DB15K, FB15K-YAGO15K). Qualitative experiments also showed that the importance of information quantified by uncertainty successfully captured data that was not useful for information retrieval. Our qualitative experiments also show that the importance of information, quantified by uncertainty, effectively captures data that is not beneficial for information retrieval.

show abstract

“…Wang et al [30] propose the TransAE, which combines self-encoder and TransE to learn MKG representation for knowledge inference. Lu et al [31] propose the Multi-modal knowledge graph representation learning model, which introduces a multi-modal knowledge alignment scheme to correlate and merge multi-modal knowledge and uses an adversarial training strategy to enhance its robustness. Ning et al [32] propose the PDRL model, which combines relational paths in the knowledge graph with entity description information to improve model performance.…”

Section: Multi-modal Knowledge Graphs Inferencementioning

confidence: 99%

“…Lu et al. [31] propose the Multi‐modal knowledge graph representation learning model, which introduces a multi‐modal knowledge alignment scheme to correlate and merge multi‐modal knowledge and uses an adversarial training strategy to enhance its robustness. Ning et al.…”

Section: Related Workmentioning

confidence: 99%

Multi‐modal knowledge graph inference via media convergence and logic rule

Feng

Zhang

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

Media convergence works by processing information from different modalities and applying them to different domains. It is difficult for the conventional knowledge graph to utilise multi‐media features because the introduction of a large amount of information from other modalities reduces the effectiveness of representation learning and makes knowledge graph inference less effective. To address the issue, an inference method based on Media Convergence and Rule‐guided Joint Inference model (MCRJI) has been proposed. The authors not only converge multi‐media features of entities but also introduce logic rules to improve the accuracy and interpretability of link prediction. First, a multi‐headed self‐attention approach is used to obtain the attention of different media features of entities during semantic synthesis. Second, logic rules of different lengths are mined from knowledge graph to learn new entity representations. Finally, knowledge graph inference is performed based on representing entities that converge multi‐media features. Numerous experimental results show that MCRJI outperforms other advanced baselines in using multi‐media features and knowledge graph inference, demonstrating that MCRJI provides an excellent approach for knowledge graph inference with converged multi‐media features.

show abstract

MMKRL: A robust embedding approach for multi-modal knowledge graph representation learning

Cited by 15 publications

References 34 publications

NativE: Multi-modal Knowledge Graph Completion in the Wild

NativE: Multi-modal Knowledge Graph Completion in the Wild

Multi-Modal Entity Alignment Using Uncertainty Quantification for Modality Importance

Multi‐modal knowledge graph inference via media convergence and logic rule

Contact Info

Product

Resources

About