Repurposing Entailment for Multi-Hop Question Answering Tasks

Trivedi, Harsh; Kwon, Hee Young; Khot, Tushar; Sabharwal, Ashish; Balasubramanian, Niranjan

doi:10.18653/v1/n19-1302

Cited by 43 publications

(55 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The task of selecting justification sentences is complex for multi-hop QA, because of the additional knowledge aggregation requirement (examples of such questions and answers are shown in Figures 1 and 2). Although various neural QA methods have achieved high performance on some of these datasets Trivedi et al, 2019;Tymoshenko et al, 2017;Seo et al, 2016;Wang and Jiang, 2016;De Cao et al, 2018;Back et al, 2018), we argue that more effort must be dedicated to explaining their inference process.…”

Section: Introductionmentioning

confidence: 86%

“…In the first category, previous works (e.g., (Trivedi et al, 2019)) have used entailment resources including labeled trained datasets such as SNLI (Bowman et al, 2015) and MultiNLI (Williams et al, 2017) to train components for selecting justification sentences for QA. Other works have explicitly focused on training sentence selection components for QA models (Min et al, 2018;Wang et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…In this work we propose an unsupervised algorithm for the selection of multi-hop justifications from unstructured knowledge bases (KB). Unlike other supervised selection methods (Dehghani et al, 2019;Bao et al, 2016;Wang et al, 2018b,a;Tran and Niedereée, 2018;Trivedi et al, 2019), our approach does not require any training data for justification selection. Unlike approaches that rely on structured KBs, which are expensive to create, (Khashabi et al, 2016;Khot et al, 2017;Khashabi et al, 2018b;Cui et al, 2017;Bao et al, 2016), our method operates over KBs of only unstructured texts.…”

Section: Introductionmentioning

confidence: 98%

See 2 more Smart Citations

Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering

Yadav

Bethard

Surdeanu

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

We propose an unsupervised strategy for the selection of justification sentences for multihop question answering (QA) that (a) maximizes the relevance of the selected sentences, (b) minimizes the overlap between the selected facts, and (c) maximizes the coverage of both question and answer. This unsupervised sentence selection method can be coupled with any supervised QA approach. We show that the sentences selected by our method improve the performance of a state-of-the-art supervised QA model on two multi-hop QA datasets: AI2's Reasoning Challenge (ARC) and Multi-Sentence Reading Comprehension (MultiRC). We obtain new state-of-the-art performance on both datasets among approaches that do not use external resources for training the QA system: 56.82% F1 on ARC (41.24% on Challenge and 64.49% on Easy) and 26.1% EM0 on MultiRC. Our justification sentences have higher quality than the justifications selected by a strong information retrieval baseline, e.g., by 5.4% F1 in MultiRC. We also show that our unsupervised selection of justification sentences is more stable across domains than a state-of-the-art supervised sentence selection method.

show abstract

Section: Introductionmentioning

confidence: 86%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

See 1 more Smart Citation

Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering

Yadav

Bethard

Surdeanu

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…These large corpora have been used as part of larger benchmark sets, e.g., GLUE (Wang et al, 2018), and have proven useful for problems beyond NLI, such as sentence representation and transfer learning (Conneau et al, 2017;Subramanian et al, 2018;Reimers and Gurevych, 2019), automated question-answering (Khot et al, 2018;Trivedi et al, 2019) and model probing (Warstadt et al, 2019;Geiger et al, 2020;Jeretic et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

OCNLI: Original Chinese Natural Language Inference

Hu¹,

Richardson²,

Xu³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Despite the tremendous recent progress on natural language inference (NLI), driven largely by large-scale investment in new datasets (e.g., SNLI, MNLI) and advances in modeling, most progress has been limited to English due to a lack of reliable datasets for most of the world's languages. In this paper, we present the first large-scale NLI dataset (consisting of ∼56,000 annotated sentence pairs) 1 for Chinese called the Original Chinese Natural Language Inference dataset (OCNLI). Unlike recent attempts at extending NLI to other languages, our dataset does not rely on any automatic translation or non-expert annotation. Instead, we elicit annotations from native speakers specializing in linguistics. We follow closely the annotation protocol used for MNLI, but create new strategies for eliciting diverse hypotheses. We establish several baseline results on our dataset using state-of-the-art pre-trained models for Chinese, and find even the best performing models to be far outpaced by human performance (∼12% absolute performance gap), making it a challenging new resource that we hope will help to accelerate progress in Chinese natural language understanding. To the best of our knowledge, this is the first humanelicited MNLI-style corpus for a non-English language.

show abstract

“…The significant advances of NLI have led researchers in many fields to use this task to solve various problems and apply it to applications that require inference between two expressions. These include question answering 45 , fact extraction 46 , generating video captions 47 , and judging textual quality 48 and etc.…”

Section: Natural Language Inferencementioning

confidence: 99%

Fake News Detection on Social Media Using A Natural Language Inference Approach

Sadeghi

Bidgoly

Amirkhani

2020

Preprint

View full text Add to dashboard Cite

Fake news detection is a challenging problem in online social media, with considerable social and political impacts. Several methods have already been proposed for the automatic detection of fake news, which are often based on the statistical features of the content or context of news. In this paper, we propose a novel fake news detection method based on Natural Language Inference (NLI) approach. Instead of using only statistical features of the content or context of the news, the proposed method exploits a human-like approach, which is based on inferring veracity using a set of reliable news. In this method, the related and similar news published in reputable news sources are used as auxiliary knowledge to infer the veracity of a given news item. We also collect and publish the first inference-based fake news detection dataset, called FNID, in two formats: the two-class version (FNID-FakeNewsNet) and the six-class version (FNID-LIAR). We use the NLI approach to boost several classical and deep machine learning models including Decision Tree, Naïve Bayes, Random Forest, Logistic Regression, k-Nearest Neighbors, Support Vector Machine, BiGRU, and BiLSTM along with different word embedding methods including Word2vec, GloVe, fastText, and BERT. The experiments show that the proposed method achieves 85.58% and 41.31% accuracies in the FNID-FakeNewsNet and FNID-LIAR datasets, respectively, which are 10.44% and 13.19% respective absolute improvements.

show abstract

Repurposing Entailment for Multi-Hop Question Answering Tasks

Cited by 43 publications

References 22 publications

Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering

Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering

OCNLI: Original Chinese Natural Language Inference

Fake News Detection on Social Media Using A Natural Language Inference Approach

Contact Info

Product

Resources

About