JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering

Sun, Yueqing; Shi, Qi; Qi, Le

doi:10.18653/v1/2022.naacl-main.372

Cited by 18 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, several research endeavors aim to amplify representation and inference capabilities by linking LLM and KG. Sun et al [29] utilized attention mechanism to facilitate interaction between all tokens of the input text and KG entities, thus augmenting the output of the language model with domain-specific knowledge from the KG.…”

Section: Kg-powered News Recommendation Modelmentioning

confidence: 99%

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Chen,

Xie,

Cui

et al. 2024

CMC

View full text Add to dashboard Cite

Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems. Traditional methods are usually difficult to learn and acquire complex semantic information in news texts, resulting in unsatisfactory recommendation results. Besides, these traditional methods are more friendly to active users with rich historical behaviors. However, they can not effectively solve the long tail problem of inactive users. To address these issues, this research presents a novel general framework that combines Large Language Models (LLM) and Knowledge Graphs (KG) into traditional methods. To learn the contextual information of news text, we use LLMs' powerful text understanding ability to generate news representations with rich semantic information, and then, the generated news representations are used to enhance the news encoding in traditional methods. In addition, multi-hops relationship of news entities is mined and the structural information of news is encoded using KG, thus alleviating the challenge of long-tail distribution. Experimental results demonstrate that compared with various traditional models, on evaluation indicators such as AUC, MRR, nDCG@5 and nDCG@10, the framework significantly improves the recommendation performance. The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation. Our code is available at https://github.com/Xuan-ZW/LKPNR.

show abstract

Section: Kg-powered News Recommendation Modelmentioning

confidence: 99%

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Chen,

Xie,

Cui

et al. 2024

CMC

View full text Add to dashboard Cite

show abstract

“…A practical KGQA model should also have strong generalization abilities to language variations and different reformulations for the same logical form. Some works [65][66][67] utilize generative models to address the issue of coverage and have achieved good performance on GrailQA.…”

Section: Robustnessmentioning

confidence: 99%

Advancements in Complex Knowledge Graph Question Answering: A Survey

Song,

Li,

Dai

et al. 2023

Electronics

View full text Add to dashboard Cite

Complex Question Answering over Knowledge Graph (C-KGQA) seeks to solve complex questions using knowledge graphs. Currently, KGQA systems achieve great success in answering simple questions, while complex questions still present challenging issues. As a result, an increasing number of novel methods have been proposed to remedy this challenge. In this survey, we proposed two mainstream categories of methods for C-KGQA, which are divided according to their use for knowledge graph representation and construction, namely, graph metric (GM)-Based Methods and graph neural network (GNN)-based methods. Additionally, we also acknowledge the influence of ChatGPT, which has prompted further research into utilizing knowledge graphs as a knowledge source to assist in answering complex questions. We also introduced methods based on pre-trained models and knowledge graph joint reasoning. Furthermore, we have compiled research achievements from the past three years to make it easier for researchers with similar interests to obtain state-of-the-art research. Finally, we discussed the resources and evaluation methods for tackling C-KGQA tasks and summarized several research prospects in this field.

show abstract

“…The two modalities are then fused in the final step to render a QA prediction. But this shallow fusion does not facilitate interactions between the two modalities, and several methods (Sun et al 2022;Zhang et al 2022) to fuse LM and KG in the earlier layers have recently been introduced as well.…”

Section: Related Workmentioning

confidence: 99%

“…They aimed to find more appropriate ways to exchange information between the two modalities, via special tokens and nodes (Zhang et al 2022) or cross-attention (Sun et al 2022). But the approaches have a modality-specific encoder GNN and the information exchange occurs only on fusion layers resulting in modest information exchange.…”

Section: Introductionmentioning

confidence: 99%

Relation-Aware Language-Graph Transformer for Question Answering

Park¹,

Choi²,

Ko³

et al. 2023

AAAI

View full text Add to dashboard Cite

Question Answering (QA) is a task that entails reasoning over natural language contexts, and many relevant works augment language models (LMs) with graph neural networks (GNNs) to encode the Knowledge Graph (KG) information. However, most existing GNN-based modules for QA do not take advantage of rich relational information of KGs and depend on limited information interaction between the LM and the KG. To address these issues, we propose Question Answering Transformer (QAT), which is designed to jointly reason over language and graphs with respect to entity relations in a unified manner. Specifically, QAT constructs Meta-Path tokens, which learn relation-centric embeddings based on diverse structural and semantic relations. Then, our Relation-Aware Self-Attention module comprehensively integrates different modalities via the Cross-Modal Relative Position Bias, which guides information exchange between relevant entities of different modalities. We validate the effectiveness of QAT on commonsense question answering datasets like CommonsenseQA and OpenBookQA, and on a medical question answering dataset, MedQA-USMLE. On all the datasets, our method achieves state-of-the-art performance. Our code is available at http://github.com/mlvlab/QAT.

show abstract

JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering

Cited by 18 publications

References 19 publications

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Advancements in Complex Knowledge Graph Question Answering: A Survey

Relation-Aware Language-Graph Transformer for Question Answering

Contact Info

Product

Resources

About