A Heuristic-driven Ensemble Framework for COVID-19 Fake News Detection

Das, Sourya Dipta; Basak, Ayan; Dutta, Saikat

doi:10.48550/arxiv.2101.03545

Cited by 3 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table-16, we have shown that with the addition of feature fusion network, the performance of the framework has improved compared to other models and achieved state of the art results on both datasets. In our earlier work [38], we had shown that the heuristic post-processing approach improves the classification accuracy on the test set significantly. However, the incorporation of uncertainty estimation improves the model performance even more.…”

Section: Ablation Study On Heuristic Post-processingmentioning

confidence: 95%

A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles

Das¹,

Basak²,

Dutta³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The significance of social media has increased manifold in the past few decades as it helps people from even the most remote corners of the world to stay connected. With the advent of technology, digital media has become more relevant and widely used than ever before and along with this, there has been a resurgence in the circulation of fake news and tweets that demand immediate attention. In this paper, we describe a novel Fake News Detection system that automatically identifies whether a news item is "real" or "fake", as an extension of our work in the CONSTRAINT COVID-19 Fake News Detection in English challenge. We have used an ensemble model consisting of pre-trained models followed by a statistical feature fusion network , along with a novel heuristic algorithm by incorporating various attributes present in news items or tweets like source, username handles, URL domains and authors as statistical feature.Our proposed framework have also quantified reliable predictive uncertainty along with proper class output confidence level for the classification task. We have evaluated our results on the COVID-19 Fake News dataset and FakeNewsNet dataset to show the effectiveness of the proposed algorithm on detecting fake news in short news content as well as in news articles. We obtained a best F1-score of 0.9892 on the COVID-19 dataset, and an F1-score of 0.9073 on the FakeNewsNet dataset.

show abstract

Section: Ablation Study On Heuristic Post-processingmentioning

confidence: 95%

A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles

Das¹,

Basak²,

Dutta³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…On the other hand, the research corresponding to the COVID-19 dataset [6] used several traditional machine learning approaches such as Decision Tree, Logistic Regression, Gradient Boosting, and Support Vector Machine (SVM). More recent approaches [7] have integrated newer technologies, including the use of pre-trained transformers such as BERT [8] and XLNet [9].…”

Section: Related Workmentioning

confidence: 99%

Explaining Misinformation Detection Using Large Language Models

Pendyala,

Hall

2024

Electronics

View full text Add to dashboard Cite

Large language models (LLMs) are a compressed repository of a vast corpus of valuable information on which they are trained. Therefore, this work hypothesizes that LLMs such as Llama, Orca, Falcon, and Mistral can be used for misinformation detection by making them cross-check new information with the repository on which they are trained. Accordingly, this paper describes the findings from the investigation of the abilities of LLMs in detecting misinformation on multiple datasets. The results are interpreted using explainable AI techniques such as Local Interpretable Model-Agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), and Integrated Gradients. The LLMs themselves are also asked to explain their classification. These complementary approaches aid in better understanding the inner workings of misinformation detection using LLMs and lead to conclusions about their effectiveness at the task. The methodology is generic and nothing specific is assumed for any of the LLMs, so the conclusions apply generally. Primarily, when it comes to misinformation detection, the experiments show that the LLMs are limited by the data on which they are trained.

show abstract

Explaining Misinformation Detection Using Large Language Models

Pendyala,

Hall

2024

Preprint

View full text Add to dashboard Cite

LLMs are a compressed repository of a vast corpus of valuable information on which they are trained. Therefore, this work hypothesizes that LLMs such as Llama, Orca, Falcon, and Mistral can be used for misinformation detection by making them cross-check new information with the repository on which they are trained. Accordingly, this paper describes the findings from the investigation of the abilities of LLMs in detecting misinformation on multiple datasets. The results are interpreted using explainable AI techniques such as LIME, SHAP, and Integrated Gradients. The LLMs themselves are also asked to explain their classification. These complementary approaches aid in better understanding the inner workings of misinformation detection using LLMs and lead to conclusions about their effectiveness at the task.

show abstract

A Heuristic-driven Ensemble Framework for COVID-19 Fake News Detection

Cited by 3 publications

References 0 publications

A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles

A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles

Explaining Misinformation Detection Using Large Language Models

Explaining Misinformation Detection Using Large Language Models

Contact Info

Product

Resources

About