Generalizing to the Future

Zhu, Yongchun; Sheng, Qiang; Cao, Juan; Li, Shuokai; Wang, Danding; Zhuang, Fuzhen

doi:10.1145/3477495.3531816

Cited by 20 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another factor contributing to poor generalisability may be also be the words that are used to derive word-level representations, where words such as named-entities are deemed important by the model but only serve to bias the model to classify based on such words. Removing such words that increase bias in datasets has been found to improve generalisability as reported by a recent study by [26]. Moreover, the use of novel statistical techniques, such as causality analysis, that give more weighting to words that tend to generalise better could also improve model generalisability [17].…”

Section: Discussionmentioning

confidence: 99%

Exploring the Generalisability of Fake News Detection Models

Hoy

Koulouri

2022

2022 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Fake news has been shown to have a growing negative impact on societies around the world, from influencing elections to spreading misinformation about vaccines. To address this problem, current research has proposed techniques for fake news detection, demonstrating promising results in lab conditions, where models tested on an unseen portion of the same dataset perform well. However, the question of the generalisability of these techniques, and their efficacy in the realworld, is less frequently evaluated. Studies that have looked at generalisability argue that models struggle to distinguish between fake and legitimate news across different topics of news, as well as across different time periods, to the ones on which they have been trained. This prompts the more fundamental question of how well fake news models generalise across news of the same topic and time period. As such, through a series of experiments, this study explores how well popular fake news detection models and features (word-level representations and linguistic cues) generalise across similar news. The first experiment reports high accuracies, when these techniques are tested on an unseen portion of the same dataset, replicating the findings in literature. However, the second experiment reveals that these techniques struggle to generalise well, suffering drops in accuracy of around 50%, when tested against different datasets of the same topic and time period. Exploring possible reasons behind such poor generalisability, the analysis points to the issue of dataset size, motivating the need for larger, more diverse datasets to become available. It also suggests that word-level representations lead to more biased, less generalisable models. Finally, the findings provide preliminary support for the effectiveness of linguistic and stylistic features, and for the potential of features beyond the word or language level, such as URL redirections and reverse image searches.

show abstract

Section: Discussionmentioning

confidence: 99%

Exploring the Generalisability of Fake News Detection Models

Hoy

Koulouri

2022

2022 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

show abstract

“…While previous studies have leveraged syntactical dependency graphs, there remains a need for deeper exploration of these graphs to extract more syntactical relations and filter out noisy connections that may introduce irrelevant information. Besides, prior biases are another factor that needs to be considered, as they can impact the generalization capacity of fake news detection [58]. However, little research has been dedicated to understanding and mitigating such biases.…”

Section: Related Work 21 Fake News Detectionmentioning

confidence: 99%

“…BERT-Emo-ENDEF [58] is a fake news detection method that introduces an entity debiasing framework (ENDEF) in the BERT-Emo model to mitigate the bias within news pieces.…”

Section: Baselinesmentioning

confidence: 99%

“…Additionally, it is crucial to address the presence of prior biases towards specific words, which has often been overlooked in previous methods. These biases arise from the statistical tendencies of neural models towards historical data and can result in an unfair viewpoint [10,45,55,58], leading to misclassification of news articles, particularly those containing fake news [11]. Figure 1(b) illustrates this issue, where preconceived notions about the emotional word "shock" and the entity word "India" can easily influence interpretation and judgment, potentially leading to the misidentification of genuine news as fake news.…”

Section: Introductionmentioning

confidence: 99%

“…Figure 1(b) illustrates this issue, where preconceived notions about the emotional word "shock" and the entity word "India" can easily influence interpretation and judgment, potentially leading to the misidentification of genuine news as fake news. Zhu et al [58] first introduced causal learning to mitigate entity bias in fake news detection, explicitly improving the generalization ability of detectors to future news data. However, we recognize that these prior biases primarily originate not only from key entities in news articles but also from significant contextual indicators such as emotional words like "shocks" in Figure 1(b).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

MSynFD: Multi-hop Syntax Aware Fake News Detection

Xiao,

Zhang,

Shi

et al. 2024

Proceedings of the ACM on Web Conference 2024

View full text Add to dashboard Cite

The proliferation of social media platforms has fueled the rapid dissemination of fake news, posing threats to our real-life society. Existing methods use multimodal data or contextual information to enhance the detection of fake news by analyzing news content and/or its social context. However, these methods often overlook essential textual news content (articles) and heavily rely on sequential modeling and global attention to extract semantic information. These existing methods fail to handle the complex, subtle twists 1 in news articles, such as syntax-semantics mismatches and prior biases, leading to lower performance and potential failure when modalities or social context are missing. To bridge these significant gaps, we propose a novel multi-hop syntax aware fake news detection (MSynFD) method, which incorporates complementary syntax information to deal with subtle twists in fake news. Specifically, we introduce a syntactical dependency graph and design a multi-hop subgraph aggregation mechanism to capture multi-hop syntax. It extends the effect of word perception, leading to effective noise filtering and adjacent relation enhancement. Subsequently, a sequential relative position-aware Transformer is designed to capture the sequential information, together with an elaborate keyword debiasing module to mitigate the prior bias. Extensive experimental results on two public benchmark datasets verify the effectiveness and superior performance of our proposed MSynFD over state-of-the-art detection models. CCS CONCEPTS• Computing methodologies → Artificial intelligence.

show abstract

KG-MFEND: an efficient knowledge graph-based model for multi-domain fake news detection

Huanxin

Liu

2023

J Supercomput

View full text Add to dashboard Cite

The widespread dissemination of fake news on social media brings adverse effects on the public and social development. Most existing techniques are limited to a single domain (e.g., medicine or politics) to identify fake news. However, many differences exist commonly across domains, such as word usage, which lead to those methods performing poorly in other domains. In the real world, social media releases millions of news pieces in diverse domains every day. Therefore, it is of significant practical importance to propose a fake news detection model that can be applied to multiple domains. In this paper, we propose a novel framework based on knowledge graphs (KG) for multi-domain fake news detection, named KG-MFEND. The model’s performance is enhanced by improving the BERT and integrating external knowledge to alleviate domain differences at the word level. Specifically, we construct a new KG that encompasses multi-domain knowledge and injects entity triples to build a sentence tree to enrich the news background knowledge. To solve the problem of embedding space and knowledge noise, we use the soft position and visible matrix in knowledge embedding. To reduce the influence of label noise, we add label smoothing to the training. Extensive experiments are conducted on real Chinese datasets. And the results show that KG-MFEND has a strong generalization capability in single, mixed, and multiple domains and outperforms the current state-of-the-art methods for multi-domain fake news detection.

show abstract

Generalizing to the Future

Cited by 20 publications

References 33 publications

Exploring the Generalisability of Fake News Detection Models

Exploring the Generalisability of Fake News Detection Models

MSynFD: Multi-hop Syntax Aware Fake News Detection

KG-MFEND: an efficient knowledge graph-based model for multi-domain fake news detection

Contact Info

Product

Resources

About