YTTREX: Crowdsourced Analysis of YouTube’s Recommender System During COVID-19 Pandemic

Abstract

Algorithmic personalization is difficult to approach because it entails studying many different user experiences, with a lot of variables outside of our control. Two common biases are frequent in experiments: relying on corporate service API and using synthetic profiles with small regards of regional and individualized profiling and personalization. In this work, we present the result of the first crowdsourced data collections of YouTube's recommended videos via YouTube Tracking Exposed (YTTREX). Our tool collects evidence of algorithmic personalization via an HTML parser, anonymizing the users. In our experiment we used a BBC video about COVID-19, taking into account 5 regional BBC channels in 5 different languages and we saved the recommended videos that were shown during each session. Each user watched the first five second of the videos, while the extension captured the recommended videos. We took into account the top-20 recommended videos for each completed session, looking for evidence of algorithmic personalization. Our results showed that the vast majority of videos were recommended only once in our experiment. Moreover, we collected evidence that there is a significant difference between the videos we could retrieve using the official API and what we collected with our extension. These findings show that filter bubbles exist and that they need to be investigated with a crowdsourced approach.

YouTube, The Great Radicalizer? Auditing and Mitigating Ideological Biases in YouTube Recommendations

“…Our audit joins prior work [21,22,23,24] that uncovers ideological bias in YouTube's recommendation systems. Our large-scale experiments find that recommendations that users receive are aligned with their ideology, and this is especially true for right-leaning users, who are recommended an increasing number of ideologically congenial videos, also especially among the political right, and moreover that the recommendations are progressively more extreme, leading to the claimed rabbit holes [20].…”

“…A likely exception is the recent intervention to mitigate the promotion of COVID-19 vaccine misinformation content. YouTube audits show that prevalence of misinformation recommendations on this topic is less than other topics [21,22,23]. We note, however, that there is an inherent conflict between the advertising-powered business model of online social media platforms that is geared towards user engagement, which is typically maximized by dialing up biases in recommendation algorithm.…”

“…As we discuss below, trained sock puppets achieve a middle ground between untrained sock puppets and real users for auditing purposes because (i) they trigger the feedback loop effect that real users (but not untrained sock puppets) experience and (ii) the observed radicalization can be attributed solely to YouTube's recommendations (which is not possible for studies of real users). In fact, recent work has adjusted this framework of trained sock puppets to audit misinformation in YouTube's recommendation system [21,22,23,24,58]. We extend this work to the broader consequential domain of ideologically biased recommendations.…”

“…Evidence as to whether YouTube's algorithm disproportionately recommends ideologically biased content is inconclusive. Whereas some studies report that the algorithm tends to recommend radical, divisive, or conspiratorial content [21,22,23,24], others provide evidence to the contrary [25,26]. These conflicting conclusions are due to subtle but crucial differences in the methodologies.…”

Nudging the Recommendation Algorithm Increases News Consumption and Diversity on YouTube

“…In this context, observers and scholars focus on ideological bias in algorithms. The worry is that algorithmic systems recommend congenial political content and direct users to radical, conspiratorial, or otherwise problematic information (4)(5)(6)(7)(8).…”