Recommender systems and the amplification of extremist content

Abstract

Policymakers have recently expressed concerns over the role of recommendation algorithms and their role in forming "filter bubbles". This is a particularly prescient concern in the context of extremist content online; these algorithms may promote extremist content at the expense of more moderate voices. In this article, we make two contributions to this debate. Firstly, we provide a novel empirical analysis of three platforms' recommendation systems when interacting with far-right content. We find that one platform-YouTube-does amplify extreme and fringe content, while two-Reddit and Gab-do not. Secondly, we contextualise these findings into the regulatory debate. There are currently few policy instruments for dealing with algorithmic amplification, and those that do exist largely focus on transparency. We argue that policymakers have yet to fully understand the problems inherent in "de-amplifying" legal, borderline content and argue that a co-regulatory approach may offer a route towards tackling many of these challenges.

Global pandemics, conflict and networks – the dynamics of international instability, infodemics and health care in the 21st century

“…In this context, extreme or novel views, opinions and sentiments are favoured and amplified (Whittaker et al, 2021). For example, one study found that fake news was more novel than real news and spread ‘farther, faster, deeper and more broadly than the truth’ (Vosoughi et al, 2018).…”

Bots influence opinion dynamics without direct human-bot interaction: the mediating role of recommender systems

“…Content is the expressed opinions of other agents ranked by the recommender system for each agent, based on the model's predicted engagement for a given piece of content. Agents decide whether to engage with the content based on two welldocumented biases, namely the bias to engage with similar content (similarity bias or homophily bias) (Mäs and Flache 2013;Dandekar et al 2013) and the bias to engage with extreme content and confident opinions (Penrod and Cutler 1995;Price and Stone 2004;Hegselmann and Krause 2015;Edelson et al 2021;Whittaker et al 2021). An engagement function is defined as:…”

“…Empirical evidence shows that misinformation and extreme online views tend to be more engaging than accurate or moderate content (Edelson et al 2021). Recommender systems seem critical in promoting extreme content over moderate ones (Whittaker et al 2021). Partisan content tends to remain confined in insulated clusters of users, thus reducing the opportunity to encounter cross-cutting content (Bakshy et al 2015).…”

Witnessing History: Reflections on Teaching in Turbulent Times

“…In quarantine, people turned to their screens for information and entertainment. Google, YouTube, and social media algorithms funneled users toward more extremist content (Whittaker et al, 2021). People fell down conspiracy-theory rabbit holes looking for answers.…”