Automatic Detection of Chinese Generated Essayss Based on Pre-trained BERT

Jin, Peng; Jing, Siyuan; Xie, Cheng

doi:10.1109/itaic54216.2022.9836571

Cited by 2 publications

(3 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the works, which are the most simi-lar to ours are those on detecting LM-generated texts in other languages (e.g. Russian, Chinese, French) (Shamardina et al, 2022;Chen et al, 2022a;Antoun et al, 2023), but they only detect LM-generated texts, and ignore any fakeness of their content. Similar to ours is also the new research on detecting ChatGPT.…”

Section: Related Workmentioning

confidence: 75%

“…Although there are generators for different languages, the existing datasets for detectors training are mostly in English (Fagni et al, 2021;Liyanage et al, 2022), with Chinese (Chen et al, 2022a) and Russian 12 (Posokhov et al, 2022;Shamardina et al, 2022) also available. However, there are no datasets with Bulgarian LM-generated texts, especially social-media-like.…”

Section: Related Workmentioning

confidence: 99%

“…There is Natural Language Processing (NLP) research on detecting LM-generated texts and TDFs for English and other languages (Jawahar et al, 2020;Fagni et al, 2021;Kowalczyk et al, 2022;Gambini et al, 2022;Stiff and Johansson, 2022;Sadiq and Ullah;Shamardina et al, 2022;Chen et al, 2022b). However, to the best of our knowledge, there is no such research for Bulgarian.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Looking for Traces of Textual Deepfakes in Bulgarian on Social Media

Temnikova,

Marinova,

Gargova

et al. 2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

Textual deepfakes can cause harm, especially on social media. At the moment, there are models trained to detect deepfake messages mainly for the English language, but no research or datasets currently exist for detecting them in most low-resource languages, such as Bulgarian. To address this gap, we explore three approaches. First, we machine translate an English-language social media dataset with bot messages into Bulgarian. However, the translation quality is unsatisfactory, leading us to create a new Bulgarian-language dataset with real social media messages and those generated by two language models (a new Bulgarian GPT-2 model -GPT-WEB-BG 1 , and ChatGPT). We machine translate it into English and test existing English GPT-2 and ChatGPT detectors on it, achieving only 0.44-0.51 accuracy. Next, we train our own classifiers on the Bulgarian dataset, obtaining an accuracy of 0.97. Additionally, we apply the classifier with the highest results to a recently released Bulgarian social media dataset with manually fact-checked messages, which successfully identifies some of the messages as generated by Language Models (LM). Our results show that the use of machine translation is not suitable for textual deepfakes detection. We conclude that combining LM text detection with fact-checking is the most appropriate method for this task, and that identifying Bulgarian textual deepfakes is indeed possible.

show abstract

Section: Related Workmentioning

confidence: 75%