On-the-fly Detection of Autogenerated Tweets

Abstract

Most previous work related to tweet classification have focused on identifying a given tweet as a spam, or to classify a Twitter user account as a spammer or a bot. In most cases the tweet classification has taken place offline, on a pre-collected dataset of tweets. In this paper we present an on-the-fly approach to classify each newly downloaded tweet as autogenerated or not. We define an autogenerated tweet (AGT) as a tweet where all or parts of the natural language content is generated automatically by a bot or other type of program.Our on-the-fly approach makes use of two classifiers. The first classifies a tweet solely based on the twitter text and the tweet metadata that comes with every tweet. It is used for tweets posted by unknown users with no available tweet history. An unknown user also triggers a batch job to start downloading the missing user timeline information. The second classifier is used for tweets posted by a user where the user timeline is downloaded and available. Initially, it will be the first classifier that handles most of the tweets. This will gradually change and after an initialization phase where we download historic data for the most active users, we reach a state where the second classifier handles a vast majority of all the tweets.A simulation using our on-the-fly detection mechanism indicates that we can handle Twitter streams with up to 68,000 unique users each day. The bottleneck is the time required to download new user timelines. The AGT detection is very accurate. In a set of 5,000 tweets we correctly classified about 98% of all AGTs using a subject-wise cross-validation.

Deepfake Detection on Social Media: Leveraging Deep Learning and FastText Embeddings for Identifying Machine-Generated Tweets

“…There is also a lack of properly labelled datasets containing human and machine-generated short text in the research community [19]. Researchers in [28], [29] used a tweet dataset containing tweets generated by a wide range of bots like cyborg, social bot, spam bot, and sockpuppet [30]. However, their dataset was human labelled and research claimed that humans are unable to identify machine-generated text.…”