Auditing radicalization pathways on YouTube

Abstract

Non-profits and the media claim there is a radicalization pipeline on YouTube. Its content creators would sponsor fringe ideas, and its recommender system would steer users towards edgier content. Yet, the supporting evidence for this claim is mostly anecdotal, and there are no proper measurements of the influence of YouTube's recommender system. In this work, we conduct a large scale audit of user radicalization on YouTube. We analyze 331, 849 videos of 360 channels which we broadly classify into: control, the Alt-lite, the Intellectual Dark Web (I.D.W.), and the Alt-right -channels in the I.D.W. and the Alt-lite would be gateways to fringe far-right ideology, here represented by Alt-right channels. Processing more than 79M comments, we show that the three communities increasingly share the same user base; that users consistently migrate from milder to more extreme content; and that a large percentage of users who consume Alt-right content now consumed Alt-lite and I.D.W. content in the past. We also probe YouTube's recommendation algorithm, looking at more than 2M million recommendations for videos and channels between May and July 2019. We find that Alt-lite content is easily reachable from I.D.W. channels via recommendations and that Alt-right channels may be reached from both I.D.W. and Alt-lite channels. Overall, we paint a comprehensive picture of user radicalization on YouTube and provide methods to transparently audit the platform and its recommender system. CCS CONCEPTS• Human-centered computing → Empirical studies in collaborative and social computing.

Algorithmic extremism: Examining YouTube's rabbit hole of radicalization

“…The study furthermore failed to find support for radicalization pathways, instead of finding that 1 The study borrows a definition for the alt-right from Anti-Defamation League: "loose segment of the white supremacist movement consisting of individuals who reject mainstream conservatism in favor of politics that embrace racist, anti-Semitic and white supremacist ideology" (pp. 2 [32]). The alt-light is defined to be a civic nationalist group rather than racial nationalism groups.…”

“…Second, YouTube is being scrutinized for how and where the recommendation algorithm directs the user traffic [17] [15]. Nevertheless, empirical evidence of YouTube's role in radicalization is insufficient [32]. There are anecdotes of a radicalization pipeline and hate group rabbit hole, but academic literature on the topic is scant, as we discuss in the next section.…”

The temporal evolution of a far-right forum

“…In addition to measuring hateful language and sentiment, some work has also examined engagement amongst extremist communities, such as post sharing [21], comment activity [22], and link sharing [5]. Elsewhere, researchers examined a sample of 154K Twitter users and the extent to which they shared pro-ISIS content, defined as tweets from known pro-ISIS accounts or other accounts that were suspended for supporting ISIS [21].…”

Trustworthy Misinformation Mitigation with Soft Information Nudging

“…Partisan-based trust in information from social media is concerning as disinformation is often partisan and more prevalent on social media [29]. Furthermore, a great deal of research and discussion shows that social media recommendation systems further complicate this problem [30]. For example, Facebook's news feed algorithm has been said to "rewarded publishers for sensationalism, not accuracy or depth 1 ."…”

“…These platforms self-proclaim that they have been created to promote free speech rights that have been taken away from them through the demonetization and removal from contemporary platforms like Twitter and YouTube. While the movement of partisan and conspiracy media from mainstream platforms to alternative platforms may stagnate misinformation flow to the widerpublic (which is still up for debate), it also creates even more extreme echo chambers, which can lead to radicalization [30].…”