Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering

Wang, Peifeng; Peng, Nanyun; Ilievski, Filip; Szekely, Pedro; Ren, Xiang

doi:10.18653/v1/2020.findings-emnlp.369

Cited by 54 publications

(48 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, pre-trained LMs have been augmented with external knowledge from commonsense knowledge bases such as ConceptNet, which provides more explicit knowledge grounding and improves their performance on downstream tasks that require reasoning abilities. Wang et al (2020b), for example, retrieve multi-hop knowledge paths from ConceptNet for fine-tuning LMs for multiple choice question answering. Chang et al (2020) and Bosselut et al (2021) incorporate knowledge paths from ConceptNet into pre-trained LMs for solving the SocialIQA task .…”

Section: Related Workmentioning

confidence: 99%

Reconstructing Implicit Knowledge with Language Models

Becker

Liang²,

Frank

2021

Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Ar

View full text Add to dashboard Cite

In this work we propose an approach for generating statements that explicate implicit knowledge connecting sentences in text. We make use of pre-trained language models which we refine by fine-tuning them on specifically prepared corpora that we enriched with implicit information, and by constraining them with relevant concepts and connecting commonsense knowledge paths. Manual and automatic evaluation of the generations shows that by refining language models as proposed, we can generate coherent and grammatically sound sentences that explicate implicit knowledge which connects sentence pairs in texts -on both in-domain and out-of-domain test data.

show abstract

Section: Related Workmentioning

confidence: 99%

Reconstructing Implicit Knowledge with Language Models

Becker

Liang²,

Frank

2021

Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Ar

View full text Add to dashboard Cite

show abstract

“…ALBERT + Path Generator (Wang et al, 2020) propose a multi-hop knowledge path generator to generate structured evidence dynamically according to the question. They use a pre-trained language model as the backbone, leveraging a large amount of unstructured knowledge stored in the language model to supplement the incompleteness of the knowledge base.…”

Section: Baselinesmentioning

confidence: 99%

Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources

Chen¹,

Ji²,

Chen³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

In order to facilitate natural language understanding, the key is to engage commonsense or background knowledge. However, how to engage commonsense effectively in question answering systems is still under exploration in both research academia and industry. In this paper, we propose a novel question-answering method by integrating multiple knowledge sources, i.e. Con-ceptNet, Wikipedia, and the Cambridge Dictionary, to boost the performance. More concretely, we first introduce a novel graph-based iterative knowledge retrieval module, which iteratively retrieves concepts and entities related to the given question and its choices from multiple knowledge sources. Afterward, we use a pre-trained language model to encode the question, retrieved knowledge and choices, and propose an answer choice-aware attention mechanism to fuse all hidden representations of the previous modules. Finally, the linear classifier for specific tasks is used to predict the answer. Experimental results on the CommonsenseQA dataset show that our method significantly outperforms other competitive methods and achieves the new state-ofthe-art. In addition, further ablation studies demonstrate the effectiveness of our graph-based iterative knowledge retrieval module and the answer choice-aware attention module in retrieving and synthesizing background knowledge from multiple knowledge sources.

show abstract

“…(2) The knowledge base is incomplete (Wang et al, 2020), which will inevitably cause the descriptive knowledge of some events cannot be obtained from the KB. Thus, the model should have the ability to obtain and encode such knowledge, even if it does not exist in the KB.…”

Section: Introductionmentioning

confidence: 99%

Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks

Cao¹,

Zuo²,

Chen³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Identifying causal relations of events is an important task in natural language processing area. However, the task is very challenging, because event causality is usually expressed in diverse forms that often lack explicit causal clues. Existing methods cannot handle well the problem, especially in the condition of lacking training data. Nonetheless, humans can make a correct judgement based on their background knowledge, including descriptive knowledge and relational knowledge. Inspired by it, we propose a novel Latent Structure Induction Network (LSIN) to incorporate the external structural knowledge into this task. Specifically, to make use of the descriptive knowledge, we devise a Descriptive Graph Induction module to obtain and encode the graph-structured descriptive knowledge. To leverage the relational knowledge, we propose a Relational Graph Induction module which is able to automatically learn a reasoning structure for event causality reasoning. Experimental results on two widely used datasets indicate that our approach significantly outperforms previous state-of-the-art methods.

show abstract

Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering

Cited by 54 publications

References 26 publications

Reconstructing Implicit Knowledge with Language Models

Reconstructing Implicit Knowledge with Language Models

Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources

Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks

Contact Info

Product

Resources

About