Synthetic Disinformation Attacks on Automated Fact Verification Systems

Abstract

Automated fact-checking is a needed technology to curtail the spread of online misinformation. One current framework for such solutions proposes to verify claims by retrieving supporting or refuting evidence from related textual sources. However, the realistic use cases for fact-checkers will require verifying claims against evidence sources that could be affected by the same misinformation. Furthermore, the development of modern NLP tools that can produce coherent, fabricated content would allow malicious actors to systematically generate adversarial disinformation for fact-checkers. In this work, we explore the sensitivity of automated factcheckers to synthetic adversarial evidence in two simulated settings: ADVERSARIAL ADDITION, where we fabricate documents and add them to the evidence repository available to the fact-checking system, and ADVERSARIAL MODIFICATION, where existing evidence source documents in the repository are automatically altered. Our study across multiple models on three benchmarks demonstrates that these systems suffer significant performance drops against these attacks. Finally, we discuss the growing threat of modern NLG systems as generators of disinformation in the context of the challenges they pose to automated fact-checkers.

Design Lessons from Building Deep Learning Disinformation Generation and Detection Solutions

“…Given the approach pursued and results obtained (accuracy of 98.99%), in this research a similar approach is considered for the detection component of the solution proposed Verma et al (2022). build the MCred (Message Credibility) framework based on CNN and BERT techniques for disinformation detection through information credibility assessment using the benefits of local and global text semantics Du, Bosselut & Manning (2022). build a deep learning-based verification framework for sensitivity evaluation of deep learning-based techniques, i.e., KGAT, CorefBERT, and MLA to adversarial generated disinformation done using GROVER.…”