Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering

Wu, Peng; Huang, Shujian; Weng, Rongxiang; Zheng, Zaixiang; Zhang, Jianbing; Yan, Xiaohui; Chen, Jiajun

doi:10.18653/v1/p19-1616

Cited by 22 publications

(13 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the proposed adapter G l (•) is a simple MLP. Mikolov, Le, and Sutskever (2013) and Wu et al (2019) pointed out the representation space of similar languages can be transferred by a linear mapping. In our scenario, which is in same language, the mapping function can transfer the general representation to task-specific representation effectively.…”

Section: Dynamic Fusion Mechanismmentioning

confidence: 99%

Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation

Weng

Huang

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

Pre-training and fine-tuning have achieved great success in natural language process field. The standard paradigm of exploiting them includes two steps: first, pre-training a model, e.g. BERT, with a large scale unlabeled monolingual data. Then, fine-tuning the pre-trained model with labeled data from downstream tasks. However, in neural machine translation (NMT), we address the problem that the training objective of the bilingual task is far different from the monolingual pre-trained model. This gap leads that only using fine-tuning in NMT can not fully utilize prior language knowledge. In this paper, we propose an Apt framework for acquiring knowledge from pre-trained model to NMT. The proposed approach includes two modules: 1). a dynamic fusion mechanism to fuse task-specific features adapted from general knowledge into NMT network, 2). a knowledge distillation paradigm to learn language knowledge continuously during the NMT training process. The proposed approach could integrate suitable knowledge from pre-trained models to improve the NMT. Experimental results on WMT English to German, German to English and Chinese to English machine translation tasks show that our model outperforms strong baselines and the fine-tuning counterparts.

show abstract

Section: Dynamic Fusion Mechanismmentioning

confidence: 99%

Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation

Weng

Huang

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

show abstract

“…This setting is an instance of domain adaptation, where a model is trained on data S, which is drawn according a source distribution, and tested on data T coming from a different target distribution. Domain adaptation over KG domains is more challenging compared to domain adaptation over single KG relations [Yu et al, 2017, Wu et al, 2019, because it is less likely for relations with similar lexicalizations to appear in the training set.…”

Section: Problem Statementmentioning

confidence: 99%

Knowledge Graph Simple Question Answering for Unseen Domains

Sidiropoulos¹,

Voskarides²,

Kanoulas³

2020

Preprint

View full text Add to dashboard Cite

Knowledge graph simple question answering (KGSQA), in its standard form, does not take into account that human-curated question answering training data only cover a small subset of the relations that exist in a Knowledge Graph (KG), or even worse, that new domains covering unseen and rather different to existing domains relations are added to the KG. In this work, we study KGSQA in a previously unstudied setting where new, unseen domains are added during test time. In this setting, question-answer pairs of the new domain do not appear during training, thus making the task more challenging. We propose a data-centric domain adaptation framework that consists of a KGSQA system that is applicable to new domains, and a sequence to sequence question generation method that automatically generates question-answer pairs for the new domain. Since the effectiveness of question generation for KGSQA can be restricted by the limited lexical variety of the generated questions, we use distant supervision to extract a set of keywords that express each relation of the unseen domain and incorporate those in the question generation method. Experimental results demonstrate that our framework significantly improves over zero-shot baselines and is robust across domains.

show abstract

“…In question answering, when KB evolves, the questions related to the new knowledge should be readily answered, as illustrated in Figure 1. However, it is difficult for a typical KBQA model to answer these questions because it has no ability to detect the relations that are not available in training (Wu et al 2019). To answer these questions, we can certainly re-train a new robust KBQA model over the entire data once the additional KB knowledge comes.…”

Section: Introductionmentioning

confidence: 99%

Incremental Knowledge Based Question Answering

Li¹,

Li²,

Nie³

2021

Preprint

View full text Add to dashboard Cite

In the past years, Knowledge-Based Question Answering (KBQA), which aims to answer natural language questions using facts in a knowledge base, has been well developed. Existing approaches often assume a static knowledge base. However, the knowledge is evolving over time in the real world. If we directly apply a fine-tuning strategy on an evolving knowledge base, it will suffer from a serious catastrophic forgetting problem. In this paper, we propose a new incremental KBQA learning framework that can progressively expand learning capacity as humans do. Specifically, it comprises a margin-distilled loss and a collaborative exemplar selection method, to overcome the catastrophic forgetting problem by taking advantage of knowledge distillation. We reorganize the SimpleQuestion dataset to evaluate the proposed incremental learning solution to KBQA. The comprehensive experiments demonstrate its effectiveness and efficiency when working with the evolving knowledge base.

show abstract

Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering

Cited by 22 publications

References 31 publications

Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation

Acquiring Knowledge from Pre-Trained Model to Neural Machine Translation

Knowledge Graph Simple Question Answering for Unseen Domains

Incremental Knowledge Based Question Answering

Contact Info

Product

Resources

About