Detecting Check-worthy Factual Claims in Presidential Debates

Abstract

Public figures such as politicians make claims about "facts" all the time. Journalists and citizens spend a good amount of time checking the veracity of such claims. Toward automatic fact checking, we developed tools to find check-worthy factual claims from natural language sentences. Specifically, we prepared a U.S. presidential debate dataset and built classification models to distinguish check-worthy factual claims from non-factual claims and unimportant factual claims. We also identified the most-effective features based on their impact on the classification models' accuracy.

Emergent: a novel data-set for stance classification

“…The choice of claims to factcheck is a task in its own right, as shown by Hassan et al (2015). Finally, the only other use of data from the Emergent project is by Liu et al (2015); however their focus was not on the NLP aspects of the task but on using Twitter data to assess the veracity of the claim, ignoring the articles and their stances curated by the journalists.…”

A Content Management Perspective on Fact-Checking

“…ClaimBuster [31,32] (whose functionalities we describe below) is quite complete; a recent vision paper [48] points toward such a comparable architecture, and FullFact also states they are working to develop a complete platform [3]. The CJ Workbench [59] is another example of grass-root initiative to automate journalistic work.…”

“…For instance, [43] presents a corpus where claims have been manually classified as verifiable and unverifiable, and [29] uses different kinds of neural networks to learn a classification model. Support Vector Machines is used in ClaimBuster [31,32] which monitors data sources such as social media, TV programs and websites, analyses the incoming stream of information and classifies claims in three categories: non-factual (e.g., opinions or subjective content); factual but not interesting (consensual, general); factual and interesting (that is, check-worthy). To train the classifier, a database of 20,000 claims annotated with these categories has been built through crowdsourcing.…”

“…ClaimBuster [31,32] attempts to match the claims against previously checked claims from trusted repositories. If no identical or similar claim can be found, Web search engines and question answering systems such as Wolfram Alpha are leveraged to obtain evidence backing or contradicting the claim.…”

An Extensible Framework for Verification of Numerical Claims

“…Existing fact checking systems are capable of detecting fact-check-worthy claims in text (Hassan et al, 2015b), returning semantically similar textual claims (Walenz et al, 2014); and scoring the truth of triples on a knowledge graph through semantic distance (Ciampaglia et al, 2015). However, neither of these are suitable for fact checking a claim made in natural language against a database.…”