Seek and you shall find? A content analysis on the diversity of five search engines’ results on political queries

Abstract

Seek and you shall find? A content analysis on the diversity of five search engines' results on political queriesSearch engines are important political news sources and should thus provide users with diverse political information -an important precondition of a well-informed citizenry. The search engines' algorithmic content selection strongly influences the diversity of the content received by the users -particularly since most users highly trust search engines and often click on only the first result. A widespread concern is that users are not informed diversely by search engines, but how far this concern applies has hardly been investigated. Our study is the first to investigate content diversity provided by five search engines on ten current political issues in Germany. The findings show that sometimes even the first result is highly diverse, but in most cases, more results must be considered to be informed diversely. This unreliability presents a serious challenge when using search engines as political news sources. Our findings call for media policy measures, for example in terms of algorithmic transparency.possible, but by no means guaranteed -particularly if a user simply clicks on the first result, as most users do (Pan et al., 2007).Below, we first describe how the filtering and sorting of search engines can affect content diversity, give an overview of the few existing empirical studies thereon, and derive our innovative measurement of content diversity. Afterwards, we present our findings and discuss their implications as well as our study's limitations. Conceptual framework How search engines influence content diversityDiversity is considered a precondition of healthy democracies (Napoli, 1999) since it is assumed to guarantee a public debate with opposing viewpoints and a well-informed citizenry, as illustrated by the "marketplace of ideas" -an idealized metaphor of public discourse (Karppinen, 2006): citizens shall freely exchange diverse ideas and viewpoints to ensure well-informed decision-making, tolerance toward other viewpoints (Jandura & Friedrich, 2014), and the stimulation of "popular wisdom" (Donohue & Glasser, 1978, p. 592). The media should contribute to it by providing diverse content (Jandura & Friedrich, 2014), stressing the importance of diversity in media policy (Just, 2009) and in communication research.Originally, the debate focused on human, journalistic gatekeepers, often revitalized by developments considered as potential threats to content diversity (e.g., concentration processes in the newspaper market (Donohue & Glasser, 1978), the introduction of commercial broadcasting in European countrues (Aslama, Hellman, & Sauri, 2004)). The rise of the Internet brought along the hope of unlimited content diversity online (European Commission, 2010, p. 30), but it quickly turned out that the processing capacities of the users limit diversity: in the information flood, they rely on gatekeepers to identify relevant information more than ever. Search engines were invented exactly for ...

Gauging reference and source bias over time: how Russia’s partially state-controlled search engine Yandex mediated an anti-regime protest event

“…A second methodological question that researchers need to address is the normative benchmark against which the performance of a search algorithm is to be evaluated. Most previous studies have chosen as benchmarks the output of one or several other search engines (Jiang, 2014;Kulshrestha et al, 2019;Mowshowitz & Kawaguchi, 2005;Steiner et al, 2020). Other studies, by contrast, have derived normative baselines from related segments of social reality, such as the coverage of traditional news media or the distribution of parties in parliament (Haim et al, 2018;Unkel & Haim, 2019).…”

“…It fostered distrust towards Yandex's algorithms, and fueled an intense debate about their alleged political bias. Not only in Russia but across the globe, the potential consequences of search engine 'bias' have recently been discussed with great concern, in academic and in public domains alike (Levchenko, 2017;Lokhov, 2019;Steiner et al, 2020;Unkel & Haim, 2019). As has been argued, search engines have developed into one of the most powerful mediators of socio-political information in the digital age (Puschmann, 2019).…”

“…By tackling this rather narrowly delimited research question, this study seeks to contribute to the filling of a broader gap in the extant literature about search engine bias in the discipline of communications. While massive efforts have been dedicated to studying search engine bias in democratic contexts (Epstein & Robertson, 2015;Granka, 2010;Kulshrestha et al, 2019;Metaxa et al, 2019;Mowshowitz & Kawaguchi, 2005;Steiner et al, 2020;Unkel & Haim, 2019), and a growing body of literature has scrutinized censorship of search engines in China (Jiang, 2014;Paltemaa et al, 2020;Vuori & Paltemaa, 2019), virtually no academic study to date has set out to investigate a potential politicization of the algorithms of Russia's partially state-controlled search engine Yandex (for an exploratory study, consider Zavadski & Toepfl, 2019). As of 2020, two of the world's five most popular Web search engines were operated by companies based in authoritarian regimes: Yandex in Russia and Baidu in China (StatCounter, 2020).…”

Searching for Representation: A sociotechnical audit of googling for members of U.S. Congress

“…Previous audits of search engines in the political context have primarily concerned themselves with whether search engines are biased. While these studies have found limited evidence of search engine political bias (Robertson et al 2018), there is some evidence of personalization of results by location (Kliman-Silver et al 2015;Robertson, Lazer, and Wilson 2018) and a lack of information diversity in political contexts (Steiner et al 2020). Metaxa et al recasts search results as "search media" and proposes longitudinal audits as a way to understand political trends (Metaxa et al 2019).…”

“…Search engines are an important part of the "online civic infrastructure" (Thorson, Xu, and Edgerly 2018) that allows voters to access political information (Dutton et al 2017;Sinclair and Wray 2015). While an informed citizenry is considered a prerequisite for an accountable democracy (Carpini and Keeter 1996), political scientists have long understood that the cost of becoming a well-informed voter is too high for most Americans (Lupia 2016).…”