Claim Detection in Biomedical Twitter Posts

Wührl, Amelie; Klinger, Roman

doi:10.18653/v1/2021.bionlp-1.15

Cited by 18 publications

(13 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…neural network 4 to predict this binary class for the remaining tweets. We further filter tweets unlikely to contain a claim using the claim detection model for tweets from Wührl and Klinger (2021). After removing duplicates and retweets, we are left with a set of 3,785 biomedical tweets containing claims with causal relations, from which we randomly sample 300 tweets for our annotation tasks.…”

Section: Data Collection and Preprocessingmentioning

confidence: 99%

CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets

Mohr¹,

Wührl²,

Klinger³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Over the course of the COVID-19 pandemic, large volumes of biomedical information concerning this new disease have been published on social media. Some of this information can pose a real danger to people's health, particularly when false information is shared, for instance recommendations on how to treat diseases without professional medical advice. Therefore, automatic fact-checking resources and systems developed specifically for the medical domain are crucial. While existing fact-checking resources cover COVID-19-related information in news or quantify the amount of misinformation in tweets, there is no dataset providing fact-checked COVID-19-related Twitter posts with detailed annotations for biomedical entities, relations and relevant evidence. We contribute CoVERT, a fact-checked corpus of tweets with a focus on the domain of biomedicine and COVID-19-related (mis)information. The corpus consists of 300 tweets, each annotated with medical named entities and relations. We employ a novel crowdsourcing methodology to annotate all tweets with fact-checking labels and supporting evidence, which crowdworkers search for online. This methodology results in moderate inter-annotator agreement. Furthermore, we use the retrieved evidence extracts as part of a fact-checking pipeline, finding that the real-world evidence is more useful than the knowledge indirectly available in pretrained language models.

show abstract

Section: Data Collection and Preprocessingmentioning

confidence: 99%

CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets

Mohr¹,

Wührl²,

Klinger³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another dataset introduced by Iskender et al (2021) includes tweets in German about climate change for claim and evidence detection. Wührl and Klinger (2021) created a dataset for biomedical Twitter claims related to COVID-19, measles, cystic fibrosis and depression. One common theme and challenge among all the datasets is the variety of claims where some types of claims (like implicit) are harder to detect than explicit ones where a typical claim structure is present.…”

Section: Text-based Approachesmentioning

confidence: 99%

MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

Cheema¹,

Hakimov²,

Sittar³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, the problem of misinformation on the web has become widespread across languages, countries, and various social media platforms. Although there has been much work on automated fake news detection, the role of images and their variety are not well explored. In this paper, we investigate the roles of image and text at an earlier stage of the fake news detection pipeline, called claim detection. For this purpose, we introduce a novel dataset, MM-Claims, which consists of tweets and corresponding images over three topics: COVID-19, Climate Change and broadly Technology. The dataset contains roughly 86 000 tweets, out of which 3400 are labeled manually by multiple annotators for the training and evaluation of multimodal models. We describe the dataset in detail, evaluate strong unimodal and multimodal baselines, and analyze the potential and drawbacks of current models.

show abstract

“…The topic of COVID-19 is nowadays popular not only in the biomedical field, but also in social science and, especially, NLP research (Verspoor et al, 2020). There already exist several datasets on COVID-19 for stance detection (Wührl and Klinger, 2021), argument mining and fact extraction/verification. For instance, in (Beck et al, 2021) the authors collect a dataset from German Twitter on people's attitude towards the government measures.…”

Section: Previous Workmentioning

confidence: 99%

RuArg-2022: Argument Mining Evaluation

Kotelnikov,

Loukachevitch,

Nikishina

et al. 2022

Preprint

View full text Add to dashboard Cite

Argumentation analysis is a field of computational linguistics that studies methods for extracting arguments from texts and the relationships between them, as well as building argumentation structure of texts. This paper is a report of the organizers on the first competition of argumentation analysis systems dealing with Russian language texts within the framework of the Dialogue conference. During the competition, the participants were offered two tasks: stance detection and argument classification. A corpus containing 9,550 sentences (comments on social media posts) on three topics related to the COVID-19 pandemic (vaccination, quarantine, and wearing masks) was prepared, annotated, and used for training and testing. The system that won the first place in both tasks used the NLI (Natural Language Inference) variant of the BERT architecture, automatic translation into English to apply a specialized BERT model, retrained on Twitter posts discussing COVID-19, as well as additional masking of target entities. This system showed the following results: for the stance detection task an F1-score of 0.6968, for the argument classification task an F1-score of 0.7404. We hope that the prepared dataset and baselines will help to foster further research on argument mining for the Russian language.

show abstract

Claim Detection in Biomedical Twitter Posts

Cited by 18 publications

References 49 publications

CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets

CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets

MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

RuArg-2022: Argument Mining Evaluation

Contact Info

Product

Resources

About