UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

Yoneda, Takuma; Mitchell, Jeff; Welbl, Johannes; Stenetorp, Pontus; Riedel, Sebastian

doi:10.18653/v1/w18-5515

Cited by 97 publications

(124 citation statements)

References 8 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…The authors in (Malon, 2018) use TF-IDF along with exact matching of the page titles with the claim's named entities. The UCL team (Yoneda et al, 2018) highlights the pages titles, and detect them in the claims. They rank pages by logistic regression and extra features like capitalization, sentence position and token matching.…”

Section: Document Retrievalmentioning

confidence: 99%

“…In order to extract evidence sentences, (Thorne et al, 2018) use a TF-IDF approach similar to their document retrieval. The UCL team (Yoneda et al, 2018) trains a logistic regression model on a heuristically set of features.…”

Section: Sentence Retrievalmentioning

confidence: 99%

“…The Papelo team (Malon, 2018) employs transformer networks with pre-trained weights (Radford et al, 2018). ESIM has been widely used among the FEVER challenge participants (Nie et al, 2019;Yoneda et al, 2018;Hanselowski et al, 2018). UNC (Nie et al, 2019), the winner of the competition, proposes a modified ESIM that takes the concatenation of the retrieved evidence sentences and claim along with ELMo embedding and three additional token-level features: Word-Net, number embedding, and semantic relatedness score from the document retrieval and sentence retrieval steps.…”

Section: Claim Verificationmentioning

confidence: 99%

See 2 more Smart Citations

BERT for Evidence Retrieval and Claim Verification

Soleimani

Monz

Worring

2020

Lecture Notes in Computer Science

118

View full text Add to dashboard Cite

Motivated by the promising performance of pre-trained language models, we investigate BERT in an evidence retrieval and claim verification pipeline for the FEVER fact extraction and verification challenge. To this end, we propose to use two BERT models, one for retrieving potential evidence sentences supporting or rejecting claims, and another for verifying claims based on the predicted evidence sets. To train the BERT retrieval system, we use pointwise and pairwise loss functions, and examine the effect of hard negative mining. A second BERT model is trained to classify the samples as supported, refuted, and not enough information. Our system achieves a new state of the art recall of 87.1 for retrieving top five sentences out of the FEVER documents consisting of 50K Wikipedia pages, and scores second in the official leaderboard with the FEVER score of 69.7.

show abstract

Section: Document Retrievalmentioning

confidence: 99%

Section: Sentence Retrievalmentioning

confidence: 99%

Section: Claim Verificationmentioning

confidence: 99%

See 1 more Smart Citation

BERT for Evidence Retrieval and Claim Verification

Soleimani

Monz

Worring

2020

Lecture Notes in Computer Science

118

View full text Add to dashboard Cite

show abstract

“…The UCL Machine Reading Group (UCL MRG) (Yoneda et al, 2018) predicts the label of each evidence-claim pair and aggregates the results via a label aggregation component.…”

Section: Shared Task Systemsmentioning

confidence: 99%

GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification

Zhou¹,

Han²,

Yang³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

147

157

View full text Add to dashboard Cite

Fact verification (FV) is a challenging task which requires to retrieve relevant evidence from plain text and use the evidence to verify given claims. Many claims require to simultaneously integrate and reason over several pieces of evidence for verification. However, previous work employs simple models to extract information from evidence without letting evidence communicate with each other, e.g., merely concatenate the evidence for processing. Therefore, these methods are unable to grasp sufficient relational and logical information among the evidence. To alleviate this issue, we propose a graph-based evidence aggregating and reasoning (GEAR) framework which enables information to transfer on a fully-connected evidence graph and then utilizes different aggregators to collect multievidence information. We further employ BERT, an effective pre-trained language representation model, to improve the performance. Experimental results on a large-scale benchmark dataset FEVER have demonstrated that GEAR could leverage multi-evidence information for FV and thus achieves the promising result with a test FEVER score of 67.10%. Our code is available at https://github. com/thunlp/GEAR.

show abstract

“…FEVER requires three sub tasks: document retrieval, evidence extraction, and answer prediction. In the previous work, the sub tasks are performed using pipelined models (Nie et al, 2019;Yoneda et al, 2018). In contrast, our approach performs evidence extraction and answer prediction simultaneously by regarding FEVER as an explainable multi-hop QA task.…”

Section: Rtementioning

confidence: 99%

Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction

Nishida¹,

Nishida²,

Nagata³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Question answering (QA) using textual sources for purposes such as reading comprehension (RC) has attracted much attention. This study focuses on the task of explainable multi-hop QA, which requires the system to return the answer with evidence sentences by reasoning and gathering disjoint pieces of the reference texts. It proposes the Query Focused Extractor (QFE) model for evidence extraction and uses multi-task learning with the QA model. QFE is inspired by extractive summarization models; compared with the existing method, which extracts each evidence sentence independently, it sequentially extracts evidence sentences by using an RNN with an attention mechanism on the question sentence. It enables QFE to consider the dependency among the evidence sentences and cover important information in the question sentence. Experimental results show that QFE with a simple RC baseline model achieves a state-of-the-art evidence extraction score on HotpotQA. Although designed for RC, it also achieves a state-of-the-art evidence extraction score on FEVER, which is a recognizing textual entailment task on a large textual database.

show abstract

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

Cited by 97 publications

References 8 publications

BERT for Evidence Retrieval and Claim Verification

BERT for Evidence Retrieval and Claim Verification

GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification

Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction

Contact Info

Product

Resources

About