Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit

Abstract

Should social media platforms intervene when communities repeatedly break rules? What actions can they consider? In light of this hotly debated issue, platforms have begun experimenting with softer alternatives to outright bans. We examine one such intervention called quarantining, that impedes direct access to and promotion of controversial communities. Specifically, we present two case studies of what happened when Reddit quarantined the influential communities r/TheRedPill (TRP) and r/The_Donald (TD). Working with over 85M Reddit posts, we apply causal inference methods to examine the quarantine's effects on TRP and TD. We find that the quarantine made it more difficult to recruit new members: new user influx to TRP and TD decreased by 79.5% and 58%, respectively. Despite quarantining, existing users' misogyny and racism levels remained unaffected. We conclude by reflecting on the effectiveness of this design friction in limiting the influence of toxic communities and discuss broader implications for content moderation.

Make Reddit Great Again: Assessing Community Effects of Moderation Interventions on r/The_Donald

“…(1) Quarantine (June 26, 2019): r/The_Donald was quarantined following repeated reports for inciting violence, including threatening US public figures. A quarantined subreddit is removed from the platform's search results and from the feed of non-subscribed users, albeit these can still access its content if they opt-in after receiving a warning message upon visit [11]. These measures intended to reduce the subreddit's visibility and to deter newcomers.…”

“…Chandrasekharan et al [12] as well as Saleem and Ruths [55] independently analyzed Reddit's bans of racist (r/CoonTown) and fat-shaming (r/fatpeoplehate) communities, finding that banning such communities significantly decreased the usage of hate speech by their former members. In a different study [11], Chandrasekharan et al also investigated the effects of quarantining offensive communities, finding that the intervention significantly reduced the number of new users joining the communities, but not their overall extent of misogyny and racism. Horta Ribeiro et al analyzed the activity of the former members of banned communities when they migrated to new platforms [33].…”

“…Firstly, interventions may produce multiple effects [39]. However, existing literature almost exclusively investigated intervention effects with respect to the activity and the toxicity (or hate speech) of the affected users [11,12,33,35,55]. Other important dimensions were overlooked, such as the political polarization and the degree of factual reporting [2].…”

“…Secondly, many different types of interventions are adopted by platform administrators, but the vast majority of studies only analyzed restrictions and bans (i.e., deplatforming) [12,33,35,55]. The effects (or lack thereof) of other interventions, such as quarantines, are unclear [11]. Thirdly, existing studies evaluated each intervention in isolation.…”

“…Thus, when a community suffers a moderation intervention, the behavior of the members of that community might change not only within that community, but also in the other communities in which they participate. However, in the case of quarantine, its effects were primarily evaluated within the community targeted by the intervention [11], and not on the other possibly affected ones.…”