Botometer 101: social bot practicum for computational social scientists

Yang, Kai-Cheng; Ferrara, Emilio; Menczer, Filippo

doi:10.1007/s42001-022-00177-5

Cited by 62 publications

(29 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that, due to limitations with the Botometer API, we were only able to subsample 500 posts per cluster per year as a rough approximation of bot activity. Our decision to use a general .70 cutoff as a delineator between likely bot and likely human account is supported by Botometer validation literature and other studies leveraging Botometer for bot detection and removal [ 56 , 57 ].…”

Section: Methodsmentioning

confidence: 99%

Using Natural Language Processing to Explore “Dry January” Posts on Twitter: Longitudinal Infodemiology Study

Russell¹,

Valdez²,

Chiang³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

Background Dry January, a temporary alcohol abstinence campaign, encourages individuals to reflect on their relationship with alcohol by temporarily abstaining from consumption during the month of January. Though Dry January has become a global phenomenon, there has been limited investigation into Dry January participants’ experiences. One means through which to gain insights into individuals’ Dry January-related experiences is by leveraging large-scale social media data (eg, Twitter chatter) to explore and characterize public discourse concerning Dry January. Objective We sought to answer the following questions: (1) What themes are present within a corpus of tweets about Dry January, and is there consistency in the language used to discuss Dry January across multiple years of tweets (2020-2022)? (2) Do unique themes or patterns emerge in Dry January 2021 tweets after the onset of the COVID-19 pandemic? and (3) What is the association with tweet composition (ie, sentiment and human-authored vs bot-authored) and engagement with Dry January tweets? Methods We applied natural language processing techniques to a large sample of tweets (n=222,917) containing the term “dry january” or “dryjanuary” posted from December 15 to February 15 across three separate years of participation (2020-2022). Term frequency inverse document frequency, k-means clustering, and principal component analysis were used for data visualization to identify the optimal number of clusters per year. Once data were visualized, we ran interpretation models to afford within-year (or within-cluster) comparisons. Latent Dirichlet allocation topic modeling was used to examine content within each cluster per given year. Valence Aware Dictionary and Sentiment Reasoner sentiment analysis was used to examine affect per cluster per year. The Botometer automated account check was used to determine average bot score per cluster per year. Last, to assess user engagement with Dry January content, we took the average number of likes and retweets per cluster and ran correlations with other outcome variables of interest. Results We observed several similar topics per year (eg, Dry January resources, Dry January health benefits, updates related to Dry January progress), suggesting relative consistency in Dry January content over time. Although there was overlap in themes across multiple years of tweets, unique themes related to individuals’ experiences with alcohol during the midst of the COVID-19 global pandemic were detected in the corpus of tweets from 2021. Also, tweet composition was associated with engagement, including number of likes, retweets, and quote-tweets per post. Bot-dominant clusters had fewer likes, retweets, or quote tweets compared with human-authored clusters. Conclusions The findings underscore the utility for using large-scale social media, such as discussions on Twitter, to study drinking reduction attempts and to monitor the ongoing dynamic needs of persons contemplating, preparing for, or actively pursuing attempts to quit or cut down on their drinking.

show abstract

Section: Methodsmentioning

confidence: 99%

Using Natural Language Processing to Explore “Dry January” Posts on Twitter: Longitudinal Infodemiology Study

Russell¹,

Valdez²,

Chiang³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…In the experimental environment, Botometer works really well. V4 has an AUC (area under the receiver operating characteristic curve) of 0.99, suggesting that the model can distinguish bot and human accounts with very high accuracy [ 55 ]. Bot score dichotomy is adopted in this study, and accounts with scores above a threshold are considered social bots.…”

Section: Methodsmentioning

confidence: 99%

“…The Botometer, however, is not flawless and may misclassify accounts owing to a variety of factors. There is a certain percentage of false negative and false positive accounts [ 56 ], depending on the data set on which the training is performed [ 55 ]. It has been pointed out that Botometer scores are imprecise when estimating bots, especially in different languages.…”

Section: Methodsmentioning

confidence: 99%

Public Opinion Manipulation on Social Media: Social Network Analysis of Twitter Bots during the COVID-19 Pandemic

Weng

Lin

2022

IJERPH

View full text Add to dashboard Cite

Social media is not only an essential platform for the dissemination of public health-related information, but also an important channel for people to communicate during the COVID-19 pandemic. However, social bots can interfere with the social media topics that humans follow. We analyzed and visualized Twitter data during the prevalence of the Wuhan lab leak theory and discovered that 29% of the accounts participating in the discussion were social bots. We found evidence that social bots play an essential mediating role in communication networks. Although human accounts have a more direct influence on the information diffusion network, social bots have a more indirect influence. Unverified social bot accounts retweet more, and through multiple levels of diffusion, humans are vulnerable to messages manipulated by bots, driving the spread of unverified messages across social media. These findings show that limiting the use of social bots might be an effective method to minimize the spread of conspiracy theories and hate speech online.

show abstract

“…We distinguished approximately 2 groups of participants: those who provided valid Twitter handles pointing to their own social media content and those who did not or refused. The latter group can be separated into three subgroups: (1) users who refused to provide a valid Twitter handle (invalid handle), (2) users who denied being Twitter users (not a Twitter user), and (3) users who did provide an existing Twitter handle, but the accounts were deemed to be bot-like as defined by a machine learning classifier [21].…”

Section: This Studymentioning

confidence: 99%

Sociodemographics and Transdiagnostic Mental Health Symptoms in SOCIAL (Studies of Online Cohorts for Internalizing Symptoms and Language) I and II: Cross-sectional Survey and Botometer Analysis

Lorenzo‐Luaces¹,

Howard²,

Edinger³

et al. 2022

JMIR Form Res

View full text Add to dashboard Cite

Background Internalizing, externalizing, and somatoform disorders are the most common and disabling forms of psychopathology. Our understanding of these clinical problems is limited by a reliance on self-report along with research using small samples. Social media has emerged as an exciting channel for collecting a large sample of longitudinal data from individuals to study psychopathology. Objective This study reported the results of 2 large ongoing studies in which we collected data from Twitter and self-reported clinical screening scales, the Studies of Online Cohorts for Internalizing Symptoms and Language (SOCIAL) I and II. Methods The participants were a sample of Twitter-using adults (SOCIAL I: N=1123) targeted to be nationally representative in terms of age, sex assigned at birth, race, and ethnicity, as well as a sample of college students in the Midwest (SOCIAL II: N=1988), of which 61.78% (1228/1988) were Twitter users. For all participants who were Twitter users, we asked for access to their Twitter handle, which we analyzed using Botometer, which rates the likelihood of an account belonging to a bot. We divided participants into 4 groups: Twitter users who did not give us their handle or gave us invalid handles (invalid), those who denied being Twitter users (no Twitter, only available for SOCIAL II), Twitter users who gave their handles but whose accounts had high bot scores (bot-like), and Twitter users who provided their handles and had low bot scores (valid). We explored whether there were significant differences among these groups in terms of their sociodemographic features, clinical symptoms, and aspects of social media use (ie, platforms used and time). Results In SOCIAL I, most individuals were classified as valid (580/1123, 51.65%), and a few were deemed bot-like (190/1123, 16.91%). A total of 31.43% (353/1123) gave no handle or gave an invalid handle (eg, entered “N/A”). In SOCIAL II, many individuals were not Twitter users (760/1988, 38.23%). Of the Twitter users in SOCIAL II (1228/1988, 61.78%), most were classified as either invalid (515/1228, 41.94%) or valid (484/1228, 39.41%), with a smaller fraction deemed bot-like (229/1228, 18.65%). Participants reported high rates of mental health diagnoses as well as high levels of symptoms, especially in SOCIAL II. In general, the differences between individuals who provided or did not provide their social media handles were small and not statistically significant. Conclusions Triangulating passively acquired social media data and self-reported questionnaires offers new possibilities for large-scale assessment and evaluation of vulnerability to mental disorders. The propensity of participants to share social media handles is likely not a source of sample bias in subsequent social media analytics.

show abstract

Botometer 101: social bot practicum for computational social scientists

Cited by 62 publications

References 51 publications

Using Natural Language Processing to Explore “Dry January” Posts on Twitter: Longitudinal Infodemiology Study

Using Natural Language Processing to Explore “Dry January” Posts on Twitter: Longitudinal Infodemiology Study

Public Opinion Manipulation on Social Media: Social Network Analysis of Twitter Bots during the COVID-19 Pandemic

Sociodemographics and Transdiagnostic Mental Health Symptoms in SOCIAL (Studies of Online Cohorts for Internalizing Symptoms and Language) I and II: Cross-sectional Survey and Botometer Analysis

Contact Info

Product

Resources

About