Multi-Source Domain Adaptation with Weak Supervision for Early Fake News Detection

Li, Yichuan; Lee, Kyumin; Kordzadeh, Nima; Faber, Brenton; Fiddes, Cameron; Chen, Elaine; Shu, Kai

doi:10.1109/bigdata52589.2021.9671592

Cited by 14 publications

(14 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…have not been trained. Our work, as well as recent research in generalizability [44], weak labeling [37], foundation models [7], and rapid fake news detection [26] falls in this period. Our generalizability experiments in Section 3 show that fine-tuned models, while having lower performance on unseen data, do have better accuracy on some subsets.…”

Section: Datasetmentioning

confidence: 78%

“…Usually, this performance degradation is detected, and a new model is trained on new labeled data. Recently, the velocity and size of new data makes obtaining labeled data quickly and at scale, very expensive [26]. Updating models during data domain shift requires relying on weak labels, authoritative sources, and hierarchical models [26,37,38].…”

Section: Motivationmentioning

confidence: 99%

“…Recently, the velocity and size of new data makes obtaining labeled data quickly and at scale, very expensive [26]. Updating models during data domain shift requires relying on weak labels, authoritative sources, and hierarchical models [26,37,38]. In such cases, a team of prediction models is pruned and updated with new training data [35].…”

Section: Motivationmentioning

confidence: 99%

See 2 more Smart Citations

Evaluating Generalizability of Fine-Tuned Models for Fake News Detection

Suprem¹,

Pu²

2022

Preprint

View full text Add to dashboard Cite

The Covid-19 pandemic has caused a dramatic and parallel rise in dangerous misinformation, denoted an 'infodemic' by the CDC and WHO. Misinformation tied to the Covid-19 infodemic changes continuously; this can lead to performance degradation of fine-tuned models due to concept drift. Degredation can be mitigated if models generalize well-enough to capture some cyclical aspects of drifted data. In this paper, we explore generalizability of pre-trained and fine-tuned fake news detectors across 9 fake news datasets. We show that existing models often overfit on their training dataset and have poor performance on unseen data. However, on some subsets of unseen data that overlap with training data, models have higher accuracy. Based on this observation, we also present KMeans-Proxy, a fast and effective method based on K-Means clustering for quickly identifying these overlapping subsets of unseen data. KMeans-Proxy improves generalizability on unseen fake news datasets by 0.1-0.2 f1-points across datasets. We present both our generalizability experiments as well as KMeans-Proxy to further research in tackling the fake news problem.

show abstract

Section: Datasetmentioning

confidence: 78%

Section: Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Generalizability of Fine-Tuned Models for Fake News Detection

Suprem¹,

Pu²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Then, a new classifier can be trained on this invariant representation for both source and target samples. Domain invariance is scalable to multiple source domains by fusing their latent representations with an adversarial encoder-discriminator framework (Li et al, 2021). For multi-source domain adaptation (MDA), classifiers for each source have different weights: static weights using distance (Li et al, 2021) or per-sample weights on l2 norm (Suprem et al, 2020).…”

Section: Multi-domain Adaptationmentioning

confidence: 99%

“…To catch up with concept drift, the classification models need to be expanded to cover a wide variety of data sets (Kaliyar et al, 2021;Li et al, 2021;Suprem and Pu, 2022), or augmented with new knowledge on true novelty such as the appearance of the Omicron variant (Pu et al, 2020). In this paper, we assume the availability of domain-specific authorative sources such as CDC and WHO that provide trusted up-to-date information on the pandemic.…”

Section: Introductionmentioning

confidence: 99%

MiDAS: Multi-integrated Domain Adaptive Supervision for Fake News Detection

Suprem¹,

Pu²

2022

Preprint

View full text Add to dashboard Cite

Covid-19 related misinformation and fake news, coined an 'infodemic', has dramatically increased over the past few years. This misinformation exhibits concept drift, where the distribution of fake news changes over time, reducing effectiveness of previously trained models for fake news detection. Given a set of fake news models trained on multiple domains, we propose an adaptive decision module to select the best-fit model for a new sample. We propose MIDAS, a multi-domain adaptative approach for fake news detection that ranks relevancy of existing models to new samples. MIDAS contains 2 components: a doman-invariant encoder, and an adaptive model selector. MIDAS integrates multiple pre-trained and fine-tuned models with their training data to create a domain-invariant representation. Then, MIDAS uses local Lipschitz smoothness of the invariant embedding space to estimate each model's relevance to a new sample. Higher ranked models provide predictions, and lower ranked models abstain. We evaluate MIDAS on generalization to drifted data with 9 fake news datasets, each obtained from different domains and modalities. MIDAS achieves new state-of-the-art performance on multi-domain adaptation for out-of-distribution fake news classification.Preprint. Under review.

show abstract

Early detection of fake news on emerging topics through weak supervision

Akdag,

Cicekli

2024

J Intell Inf Syst

View full text Add to dashboard Cite

Multi-Source Domain Adaptation with Weak Supervision for Early Fake News Detection

Cited by 14 publications

References 18 publications

Evaluating Generalizability of Fine-Tuned Models for Fake News Detection

Evaluating Generalizability of Fine-Tuned Models for Fake News Detection

MiDAS: Multi-integrated Domain Adaptive Supervision for Fake News Detection

Early detection of fake news on emerging topics through weak supervision

Contact Info

Product

Resources

About