Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering

Banerjee, Pratyay; Baral, Chitta

doi:10.18653/v1/2020.emnlp-main.11

Cited by 32 publications

(45 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table 9, we observe that self-talk significantly hurts performance for this dataset. On the other hand, contrastive explanations are found to be useful and approach the zero-shot performance of the stateof-the-art, which uses ConceptNet (Banerjee and Baral, 2020b). These results demonstrate that the set of contrastive prompts are generalizable to other commonsense reasoning datasets, and that while our contrastive prompts are limited to contrasting two answer choices at a time, the framework can be easily extended to tasks with multiple foils.…”

Section: Generalizability Of Promptsmentioning

confidence: 67%

See 1 more Smart Citation

Prompting Contrastive Explanations for Commonsense Reasoning Tasks

Paranjape¹,

Michael²,

Ghazvininejad³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit. Large pretrained language models (PLMs) can achieve near-human performance on such tasks, while providing little human-interpretable evidence of the underlying reasoning they use. In this work, we show how to use these same models to generate such evidence: inspired by the contrastive nature of human explanations, we use PLMs to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer (for example, peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts are usually salty while raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins are sweet). Conditioning model decisions on these explanations improves performance on two commonsense reasoning benchmarks, as compared to previous non-contrastive alternatives. These explanations are also judged by humans to be more relevant for solving the task, and facilitate a novel method to evaluate explanation faithfulness.

show abstract

Section: Generalizability Of Promptsmentioning

confidence: 67%

“…Table 9: Zero-shot test performance on Common-senseQA for baselines as well as contrastive models which ensemble fact/foil pairs by voting (V) and maximum margin (MM). The best reported unsupervised performance (Banerjee and Baral, 2020b) uses Con-ceptNet, which was used to construct the dataset.…”

Section: Discussionmentioning

confidence: 99%

Prompting Contrastive Explanations for Commonsense Reasoning Tasks

Paranjape¹,

Michael²,

Ghazvininejad³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

show abstract

“…There are different ways to frame the task of answering questions by machine reading, including sentence retrieval (Momtazi and Klakow, 2015), multi-hop reasoning (Khot et al, 2020), and reasoning about multiple paragraphs or documents at the same time (Dua et al, 2019;Cao et al, 2019). Recent work has considered the development of reasoning-based QA systems (Weber et al, 2019) as well as the integration of external (Banerjee and Baral, 2020) and commonsense knowledge tured knowledge graphs. A prominent example is the series of Question Answering over Linked Data (QALD) evaluation campaigns (Usbeck et al, 2018), now in its 9th edition and going back to 2011.…”

Section: Related Workmentioning

confidence: 99%

BiQuAD: Towards QA based on deeper text understanding

Grimm¹,

Cimiano

2021

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

View full text Add to dashboard Cite

Recent question answering and machine reading benchmarks frequently reduce the task to one of pinpointing spans within a certain text passage that answers the given question. Typically, these systems are not required to actually understand the text on a deeper level that allows for more complex reasoning on the information contained. We introduce a new dataset called BiQuAD that requires deeper comprehension in order to answer questions in both extractive and deductive fashion. The dataset consist of 4, 190 closed-domain texts and a total of 99, 149 question-answer pairs. The texts are synthetically generated soccer match reports that verbalize the main events of each match. All texts are accompanied by a structured Datalog program that represents a (logical) model of its information. We show that state-of-the-art QA models do not perform well on the challenging long form contexts and reasoning requirements posed by the dataset. In particular, transformer based state-of-theart models achieve F 1 -scores of only 39.0. We demonstrate how these synthetic datasets align structured knowledge with natural text and aid model introspection when approaching complex text understanding.

show abstract

“…There are different ways to frame the task of answering questions by machine reading, including sentence retrieval (Momtazi and Klakow, 2015), multi-hop reasoning (Khot et al, 2020), and reasoning about multiple paragraphs or documents at the same time (Dua et al, 2019;. Recent work has considered the development of reasoning-based QA systems (Weber et al, 2019) as well as the integration of external (Banerjee and Baral, 2020) and commonsense knowledge (Dua et al, 2019) focuses on complex reasoning tasks in form of both, open and closed domain, questions. The task combines the challenge of extractive QA with testing a models ability to perform limited numerical reasoning, e.g.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

2021

View full text Add to dashboard Cite

Recently, natural language processing (NLP) has had increasing success and produced extensive industrial applications. Despite being sufficient to enable these applications, current NLP systems often ignore the social part of language, e.g., who says it, in what context, for what goals. In this talk, we take a closer look at social factors in language via a new theory taxonomy, and its interplay with computational methods via two lines of work. The first one studies what makes language persuasive by introducing a semi-supervised method to leverage hierarchical structures in text to recognize persuasion strategies in good-faith requests. The second part demonstrates how various structures in conversations can be utilized to generate better summaries for everyday interaction. We conclude by discussing several open-ended questions towards how to build socially aware language technologies, with the hope of getting closer to the goal of human-like language understanding.

show abstract

Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering

Cited by 32 publications

References 51 publications

Prompting Contrastive Explanations for Commonsense Reasoning Tasks

Prompting Contrastive Explanations for Commonsense Reasoning Tasks

BiQuAD: Towards QA based on deeper text understanding

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

Contact Info

Product

Resources

About