Social Botomics: A Systematic Ensemble ML Approach for Explainable and Multi-Class Bot Detection

Abstract

OSN platforms are under attack by intruders born and raised within their own ecosystems. These attacks have multiple scopes from mild critiques to violent offences targeting individual or community rights and opinions. Negative publicity on microblogging platforms, such as Twitter, is due to the infamous Twitter bots which highly impact posts’ circulation and virality. A wide and ongoing research effort has been devoted to develop appropriate countermeasures against emerging “armies of bots”. However, the battle against bots is still intense and unfortunately, it seems to lean on the bot-side. Since, in an effort to win any war, it is critical to know your enemy, this work aims to demystify, reveal, and widen inherent characteristics of Twitter bots such that multiple types of bots are recognized and spotted early. More specifically in this work we: (i) extensively analyze the importance and the type of data and features used to generate ML models for bot classification, (ii) address the open problem of multi-class bot detection, identifying new types of bots, and share two new datasets towards this objective, (iii) provide new individual ML models for binary and multi-class bot classification and (iv) utilize explainable methods and provide comprehensive visualizations to clearly demonstrate interpretable results. Finally, we utilize all of the above in an effort to improve the so called Bot-Detective online service. Our experiments demonstrate high accuracy, explainability and scalability, comparable with the state of the art, despite multi-class classification challenges.

Explainable Artificial Intelligence in CyberSecurity: A Survey

“…Bot-Detective [203] is an explainable Twitter bot detection service with crowdsourcing functionalities that uses LIME. LIME is also used in JITBot [204], An Explainable Just-In-Time Defect Prediction Bot, and in [205], a bot-type classification schema. SHAP and LIME are used in [206] for game BOT detection, while in [207], the authors used a Decision Tree model, Explainable by definition, for automatic detection on Twitter with a particular case study on posts about COVID-19.…”

Botometer 101: social bot practicum for computational social scientists

“…The accuracy of the model may further decay when dealing with new accounts different from those in the training datasets. These accounts might come from a different context, use different languages other than English [ 52 , 53 ], or show novel behavioral patterns [ 34 , 45 , 54 ]. These limitations are inevitable for all supervised machine learning algorithms, and are the reasons why Botometer has to be upgraded routinely.…”

Botometer 101: Social bot practicum for computational social scientists

“…The accuracy of the model may further decay when dealing with new accounts different from those in the training datasets. These accounts might come from a different context, use different languages other than English [48,49], or show novel behavioral patterns [43,34,50]. These limitations are inevitable for all supervised machine learning algorithms, and are the reasons why Botometer has to be upgraded routinely.…”