Predicting the Volume of Response to Tweets Posted by a Single Twitter Account

Abstract

Social media users, including organizations, often struggle to acquire the maximum number of responses from other users, but predicting the responses that a post will receive before publication is highly desirable. Previous studies have analyzed why a given tweet may become more popular than others, and have used a variety of models trained to predict the response that a given tweet will receive. The present research addresses the prediction of response measures available on Twitter, including likes, replies and retweets. Data from a single publisher, the official US Navy Twitter account, were used to develop a feature-based model derived from structured tweet-related data. Most importantly, a deep learning feature extraction approach for analyzing unstructured tweet text was applied. A classification task with three classes, representing low, moderate and high responses to tweets, was defined and addressed using four machine learning classifiers. All proposed models were symmetrically trained in a fivefold cross-validation regime using various feature configurations, which allowed for the methodically sound comparison of prediction approaches. The best models achieved F1 scores of 0.655. Our study also used SHapley Additive exPlanations (SHAP) to demonstrate limitations in the research on explainable AI methods involving Deep Learning Language Modeling in NLP. We conclude that model performance can be significantly improved by leveraging additional information from the images and links included in tweets.

Automated Detection of Leadership Qualities Using Textual Data at the Message Level

“…The only case in which DL methods provided low performance was that of a pre-trained Roberta large model without any training on the data set in question. By comparing our results with those for similar methods applied to other Twitter text analysis domains, such as sentiment analysis [24] or the prediction of user response to Twitter posts [24] we find that the order of performance of the used methods agrees with those from other research: recent DL methods fine-tuned in a supervised manner on problem-specific data provide the highest prediction quality.…”

“…Twitter language is also known to have both oral and written language characteristics as debated by Wikström in [22], and Garber in [23], and the language of tweets addressed to USNTA demonstrates numerous elements of military specifics and jargon, for example, the use of acronyms such as "BZ" (Bravo Zulu), "SAPR survey" (Sexual Assault Prevention and Response survey), or "TADMUS" (tactical decision-making under stress). In this research, we have used tweets published in a previous Twitter-related study regarding sentiment analysis by Fiok et al [24].…”

Pinpointing Fine-Grained Relationships between Hateful Tweets and Replies

“…Beyond single tweet classification, tweet popularitylikes, retweets, number of replies, etc.-has been studied using account information (Matsumoto et al 2019;Fiok et al 2020) and linguistic information (Wang, Chen, and Kan 2012). Hate and counter hate have also been studied beyond single tweets.…”

NLPOP: a Dataset for Popularity Prediction of Promoted NLP Research on Twitter

“…Predicting popularity of messages is a straightforward task from the perspective of machine learning and has been framed both as a regression (Lampos et al, 2014) and classification problem (Hong et al, 2011;Jenders et al, 2013;Subramanian et al, 2018;Fiok et al, 2020), while work on information cascades (Zhao et al, 2015;Li et al, 2017;Zhou et al, 2021) focuses on modeling the entire lifetime of a post as a point process.…”