CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media

Barrón-Cedeño, Alberto; Elsayed, Tamer; Nakov, Preslav; Martino, Giovanni Da San; Hasanain, Maram; Suwaileh, Reem; Haouari, Fatima

doi:10.1007/978-3-030-45442-5_65

Cited by 29 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar approach was adopted for a related task, e.g., it was used to obtain annotated training and testing data for the Check-Worthiness task of the CLEF Check-That! Lab (Atanasova et al, 2018Barrón-Cedeño et al, 2020).…”

mentioning

confidence: 99%

That is a Known Lie: Detecting Previously Fact-Checked Claims

Shaar¹,

Babulkov²,

Martino³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

The recent proliferation of "fake news" has triggered a number of responses, most notably the emergence of several manual fact-checking initiatives. As a result and over time, a large number of fact-checked claims have been accumulated, which increases the likelihood that a new claim in social media or a new statement by a politician might have already been factchecked by some trusted fact-checking organization, as viral claims often come back after a while in social media, and politicians like to repeat their favorite statements, true or false, over and over again. As manual fact-checking is very time-consuming (and fully automatic fact-checking has credibility issues), it is important to try to save this effort and to avoid wasting time on claims that have already been fact-checked. Interestingly, despite the importance of the task, it has been largely ignored by the research community so far. Here, we aim to bridge this gap. In particular, we formulate the task and we discuss how it relates to, but also differs from, previous work. We further create a specialized dataset, which we release to the research community. Finally, we present learning-to-rank experiments that demonstrate sizable improvements over state-of-the-art retrieval and textual similarity approaches.

show abstract

mentioning

confidence: 99%

That is a Known Lie: Detecting Previously Fact-Checked Claims

Shaar¹,

Babulkov²,

Martino³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…This has encouraged the development of AI solutions, e.g., as part of shared tasks such as the CLEF CheckThat! lab 2018-2021 Elsayed et al, 2019;Barrón-Cedeño et al, 2020;Nakov et al, 2021a], and inside dedicated fact-checking organizations such as Full Fact. 12 The problem is widely tackled as a ranking one, where the system has to produce a check-worthiness scores.…”

Section: Finding Claims Worth Fact-checkingmentioning

confidence: 99%

“…The task was also featured in the CLEF CheckThat! Lab [Barrón-Cedeño et al, 2020;Nakov et al, 2021a]. Vo and Lee [2020] explored a multi-modal setup, where tweets with claims about images were matched against the Fauxtography section of Snopes.…”

Section: Detecting Previously Fact-checked Claimsmentioning

confidence: 99%

Automated Fact-Checking for Assisting Human Fact-Checkers

Nakov¹,

Corney²,

Hasanain

et al. 2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

The reporting and the analysis of current events around the globe has expanded from professional, editor-lead journalism all the way to citizen journalism. Nowadays, politicians and other key players enjoy direct access to their audiences through social media, bypassing the filters of official cables or traditional media. However, the multiple advantages of free speech and direct communication are dimmed by the misuse of media to spread inaccurate or misleading claims. These phenomena have led to the modern incarnation of the fact-checker --- a professional whose main aim is to examine claims using available evidence and to assess their veracity. Here, we survey the available intelligent technologies that can support the human expert in the different steps of her fact-checking endeavor. These include identifying claims worth fact-checking, detecting relevant previously fact-checked claims, retrieving relevant evidence to fact-check a claim, and actually verifying a claim. In each case, we pay attention to the challenges and the potential impact on real-world fact-checking.

show abstract

“…Political Speeches For political speeches, the most studied datasets come from the Clef Check-That! shared tasks (Atanasova et al, 2018;Elsayed et al, 2019;Barrón-Cedeño et al, 2020) and ClaimRank (Jaradat et al, 2018). The data consist of transcripts of political debates and speeches where each sentence has been annotated by an independent news or fact-checking organization for whether or not the statement should be checked for veracity.…”

Section: Claim Check-worthiness Detectionmentioning

confidence: 99%

“…There are multiple isolated lines of research which have studied variations of this problem. Figure 1 provides examples from three tasks which are studied in this work: rumour detection on Twitter (Zubiaga et al, 2016, check-worthiness ranking in political debates and speeches (Atanasova et al, 2018;Elsayed et al, 2019;Barrón-Cedeño et al, 2020), and citation needed detection on Wikipedia (Redi et al, 2019). Each task is concerned with a shared underlying problem: detecting claims which war-rant further verification.…”

Section: Introductionmentioning

confidence: 99%

Claim Check-Worthiness Detection as Positive Unlabelled Learning

Wright¹,

Augenstein²

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

As the first step of automatic fact checking, claim check-worthiness detection is a critical component of fact checking systems. There are multiple lines of research which study this problem: check-worthiness ranking from political speeches and debates, rumour detection on Twitter, and citation needed detection from Wikipedia. To date, there has been no structured comparison of these various tasks to understand their relatedness, and no investigation into whether or not a unified approach to all of them is achievable. In this work, we illuminate a central challenge in claim check-worthiness detection underlying all of these tasks, being that they hinge upon detecting both how factual a sentence is, as well as how likely a sentence is to be believed without verification. As such, annotators only mark those instances they judge to be clear-cut check-worthy. Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning that finds instances which were incorrectly labelled as not check-worthy. In applying this, we outperform the state of the art in two of the three tasks studied for claim check-worthiness detection in English.

show abstract

CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media

Cited by 29 publications

References 20 publications

That is a Known Lie: Detecting Previously Fact-Checked Claims

That is a Known Lie: Detecting Previously Fact-Checked Claims

Automated Fact-Checking for Assisting Human Fact-Checkers

Claim Check-Worthiness Detection as Positive Unlabelled Learning

Contact Info

Product

Resources

About