Bots Amplify and Redirect Hate Speech in Online Discourse About Racism During the COVID-19 Pandemic

Abstract

Online talk about racism has been salient throughout the COVID-19 pandemic. Yet while such social media conversations reflect existing tensions in the offline world, the same discourse has also become a target for information operations aiming to heighten social divisions. This article examines Twitter discussions of racism in the first and sixth months since COVID-19 was accorded pandemic status by the World Health Organization and uncovers dynamic associations with bot activity and hate speech. Humans initially constituted the most hateful accounts in online conversations about racism in March, but in August, bots dominated hate speech. Over time, greater bot activity likewise amplified levels of hate speech a week later. Moreover, while discourse about racism in March primarily featured an organic focus on racial identities like Asian and Chinese, we further observed a bot-dominated focus in August toward political identities like president, Democrat, and Republican. Although hate speech targeting Asian groups remained present among racism discussions in August, these findings suggest a bot-fueled redirection from focusing on racial groups at the onset of the pandemic to targeting politics closer to the 2020 US elections. This work enhances understanding of the complexity of racism discussions during the pandemic, its vulnerability to manipulation through information operations, and the large-scale quantitative study of inorganic hate campaigns in online social networks.

Libraries Combating Disinformation: From the Front Line to the Long Game

“…Much research on mis/dis/mal information emphasizes the networked and sociotechnical effects of problematic content. Social network studies tend to emphasize the role platforms and content producers play in amplifying false claims (Donovan 2020;Freelon and Lokot 2020;Freelon and Wells 2020;Uyheng, Bellutta, and Carley 2022). Sociotechnical researchers rely on more qualitative methods and theoretical arguments to better understand why false claims become believable (Marwick 2018;Tripodi 2018;Anderson 2021;Marwick and Partin 2022;).…”

“…Social network studies seek to better understand the systems that enable the spread of bogus claims. As such, this research tends to emphasize the role platforms play in creating an infrastructure ripe for misuse, often relying on web scraping and data analytics to map out how bad actors exploit those systems to amplify and circulate misinformation (Donovan 2020;Freelon and Lokot 2020;Freelon and Wells 2020;Ognyanova et al 2020;Uyheng, Bellutta, and Carley 2022;Chen et al 2021;Nisbet, Mortenson, and Li 2021). As research demonstrates, the most effective misinformation campaigns "trade up the chain"-planting stories in local/smaller news outlets insufficiently capable to fact-check claims with the goal of having the story amplified by a larger news outlet (Marwick and Lewis 2017).…”

Implicit Associations in Computer-Mediated Communication

“…In the early days of the Internet, some scholars theorized that virtual environments might provide an escape from the boundaries of biases and discrimination, and that a form of “identity tourism” might enable individuals to explore different identities in online environments, especially in gaming contexts (Daniels, 2013; Nakamura, 2002; Turkle, 1997). However, research shows that bias and discrimination exist online, and that they are perpetuated in novel forms, such as cyber-racism (Bliuc et al, 2018) and bot-amplified hate speech (Uyheng et al., 2022).…”

Ecologies of Violence on Social Media: An Exploration of Practices, Contexts, and Grammars of Online Harm

“…First, there is the research that examines how platforms enact violence inside their digital ecosystems. For instance, studies on how specific populations are deplatformed (e.g., Blunt et al, 2021), how violent content is magnified through recommendation systems (e.g., Wood, 2017), how automated accounts are created to amplify and redirect hateful speech (e.g., Uyheng et al, 2022), or how specific affordances reinforce racial and gendered violence (e.g., Yee et al, 2021). Second, there is research focused on how platforms perpetuate violence outside their digital environments.…”