Transformer-Based Language Model Fine-Tuning Methods for COVID-19 Fake News Detection

Chen, Ben; Chen, Bin; Gao, Dehong; Chen, Qijin; Huo, Chengfu; Meng, Xiaonan; Ren, Weijun; Zhou, Yang

doi:10.1007/978-3-030-73696-5_9

Cited by 29 publications

(14 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To quantify our datasets' contributions, we designed experiment setups wherein we trained RoBERTa base (Liu et al, 2019) for paraphrase detection on a combination of TwitterPPDB and our datasets as training data. RoBERTa was chosen for its generality, as it is a commonly used model in current NLP work and benchmarking, and currently achieves SOTA or near-SOTA results on a majority of NLP benchmark tasks (Wang et al, 2019(Wang et al, , 2020; Chen et al, 2021).…”

Section: Methodsmentioning

confidence: 99%

Improving Paraphrase Detection with the Adversarial Paraphrasing Task

Nighojkar¹,

Licato²

2021

Preprint

View full text Add to dashboard Cite

If two sentences have the same meaning, it should follow that they are equivalent in their inferential properties, i.e., each sentence should textually entail the other. However, many paraphrase datasets currently in widespread use rely on a sense of paraphrase based on word overlap and syntax. Can we teach them instead to identify paraphrases in a way that draws on the inferential properties of the sentences, and is not over-reliant on lexical and syntactic similarities of a sentence pair? We apply the adversarial paradigm to this question, and introduce a new adversarial method of dataset creation for paraphrase identification: the Adversarial Paraphrasing Task (APT), which asks participants to generate semantically equivalent (in the sense of mutually implicative) but lexically and syntactically disparate paraphrases. These sentence pairs can then be used both to test paraphrase identification models (which get barely random accuracy) and then improve their performance. To accelerate dataset generation, we explore automation of APT using T5, and show that the resulting dataset also improves accuracy. We discuss implications for paraphrase detection and release our dataset in the hope of making paraphrase detection models better able to detect sentence-level meaning equivalence.

show abstract

Section: Methodsmentioning

confidence: 99%

Improving Paraphrase Detection with the Adversarial Paraphrasing Task

Nighojkar¹,

Licato²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Within various studies and research, apart from tokenisation and stopword removal, authors have performed removal of HTTP URLs special characters [3][4] [5]. In the study [6], the authors, in addition to the traditional preprocessing techniques, data augmentation using the back translation technique to increase the existing data is performed.…”

Section: A Textual Data Preprocessingmentioning

confidence: 99%

A Novel Stance based Sampling for Imbalanced Data

Agarwal¹,

Rana²,

Jariwala³

et al. 2022

IJACSA

View full text Add to dashboard Cite

While the world is suffering from coronavirus pandemic (COVID-19), a parallel battle with Infodemic, the proliferation of fake news online is also taking place. The spread of fake news during this global pandemic COVID-19 has dangerous consequences. This is the driving force behind this study. Relying on incorrect information obtained from the internet or social media can be fatal. According to a World Health Organization survey, at least 800 people have lost their lives because of COVID-19 misinformation during this time, highlighting the accurate automated classification of fake news. However, the data at disposal for classification is imbalanced. The Internet has a vast repository of authentic healthcare news, whereas Fake News on COVID-19 healthcare is not abundant. This imbalance leads to incorrect classification. The paper studies alternative approaches to text sampling. In this paper, we propose a stance based sampling method for balancing news data. The disparity between the title and content of news items is utilized to sample data points selectively and rectify the imbalance. The key findings are that the proposed stance-based sampling strategies enhance categorisation task performance consistently for varying degrees of imbalance. The proposed techniques can better detect misleading news in the health care sector.

show abstract

“…We used three transformer models: m-BERT, Bangla-BERT, and XLM-R on BEmoC. In recent years transformer is being used extensively for classification tasks to achieve stateof-the-art results (Chen et al, 2021). The models are culled from the Huggingface 2 transformers library and fine-tuned on the emotion corpus by using Ktrain(Maiya, 2020) package.…”

Section: Transformer Modelsmentioning

confidence: 99%

Emotion Classification in a Resource Constrained Language Using Transformer-based Approach

Das¹,

Sharif²,

Hoque³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Rese

View full text Add to dashboard Cite

Although research on emotion classification has significantly progressed in highresource languages, it is still infancy for resource-constrained languages like Bengali. However, unavailability of necessary language processing tools and deficiency of benchmark corpora makes the emotion classification task in Bengali more challenging and complicated. This work proposes a transformer-based technique to classify the Bengali text into one of the six basic emotions: anger, fear, disgust, sadness, joy, and surprise. A Bengali emotion corpus consists of 6243 texts is developed for the classification task. Experimentation carried out using various machine learning (LR, RF, MNB, SVM), deep neural networks (CNN, BiLSTM, CNN+BiLSTM) and transformer (Bangla-BERT, m-BERT, XLM-R) based approaches. Experimental outcomes indicate that XLM-R outdoes all other techniques by achieving the highest weighted f 1 -score of 69.73% on the test data. The dataset is publicly available at https://github.com/omar-sharif03/ NAACL-SRW-2021.

show abstract

Transformer-Based Language Model Fine-Tuning Methods for COVID-19 Fake News Detection

Cited by 29 publications

References 8 publications

Improving Paraphrase Detection with the Adversarial Paraphrasing Task

Improving Paraphrase Detection with the Adversarial Paraphrasing Task

A Novel Stance based Sampling for Imbalanced Data

Emotion Classification in a Resource Constrained Language Using Transformer-based Approach

Contact Info

Product

Resources

About