How search engines disseminate information about COVID-19 and why they should do better

Abstract

Access to accurate and up-to-date information is essential for individual and collective decision making, especially at times of emergency. On February 26, 2020, two weeks before the World Health Organization (WHO) officially declared the COVID-19's emergency a "pandemic," we systematically collected and analyzed search results for the term "coronavirus" in three languages from six search engines. We found that different search engines prioritize specific categories of information sources, such as governmentrelated websites or alternative media. We also observed that source ranking within the same search engine is subjected to randomization, which can result in unequal access to information among users.

Advances in Bias and Fairness in Information Retrieval

“…To collect data, we utilized a set of virtual agents -that is software simulating user browsing behavior (e.g., scrolling web pages and entering queries) and recording its outputs. The benefits of this approach, which extends algorithmic auditing methodology introduced by Haim et al [24], is that it allows controlling for personalization [25] and randomization [35] factors influencing outputs of web search. In contrast to human actors, virtual agents can be easily synchronized (i.e., to isolate the effect of time at which the search actions are conducted) and deployed in a controlled environment (e.g., a network of virtual machines using the same IP range, the same type of operating system (OS) and the same browsing software) to limit the effects of personalization that might lead to skewed outputs.…”

Gauging reference and source bias over time: how Russia’s partially state-controlled search engine Yandex mediated an anti-regime protest event

“…This overlap increased to 92% for Google and 86% for Yandex when search histories and cookies were deleted on the participants' computers in Moscow. Against this backdrop, and with reference to recent research that indicates that both Google and Yandex randomize significant proportions of their results (Makhortykh et al, 2020), we consider the results collected for this study as broadly representative of results obtained on standard Moscow computers.…”

Auditing Source Diversity Bias in Video Search Results Using Virtual Agents

“…Our decision to rely on the top-10 results only is motivated by the fact that users tend to pay the most attention to the first few results -i.e., those on the first results page [27]. A comparison of search results by browser has demonstrated that there are no major between-browser differences -a finding in contrast with those observed for text search results [23] -thus for the analysis we have proceeded aggregating the results for both browsers.…”

“…One form of retrieval bias, which the current paper focuses on, is source diversity bias. Originally discussed in the context of search engines' tendency to prioritize web pages with the highest number arXiv:2106.02715v1 [cs.IR] 4 Jun 2021 of visitors [16], source diversity bias is currently investigated in the context of prioritization of certain categories of sources in response to particular types of queries (e.g., [23]). A disproportionate visibility of specific types of web resources can diminish overall quality of search results [6] and provide unfair advantage to companies and individuals that own specific search engines [16] -e.g., through own-content bias, -or direct most of the traffic to a handful of well-established sources, a phenomenon also known as search concentration [19].…”