Content Bias in Online Health Search

Abstract

Search engines help people answer consequential questions. Biases in retrieved and indexed content (e.g., skew toward erroneous outcomes that represent deviations from reality), coupled with searchers' biases in how they examine and interpret search results, can lead people to incorrect answers. In this article, we seek to better understand biases in search and retrieval, and in particular those affecting the accuracy of content in search results, including the search engine index, features used for ranking, and the formulation of search queries. Focusing on the important domain of online health search, this research broadens previous work on biases in search to examine the role of search systems in contributing to biases. To assess bias, we focus on questions about medical interventions and employ reliable ground truth data from authoritative medical sources. In the course of our study, we utilize large-scale log analysis using data from a popular Web search engine, deep probes of result lists on that search engine, and crowdsourced human judgments of search result captions and landing pages. Our findings reveal bias in results, amplifying searchers' existing biases that appear evident in their search activity. We also highlight significant bias in indexed content and show that specific ranking signals and specific query terms support bias. Both of these can degrade result accuracy and increase skewness in search results. Our analysis has implications for bias mitigation strategies in online search systems, and we offer recommendations for search providers based on our findings.

Perspective-based search: a new paradigm for bursting the information bubble

“…Also, another problem may happen when the user is in a minority by biasing the user's search results (according to information such as the user's social connections and geographical location) towards what the user is reluctant to (unless an exhaustive profile on the search topic is available indicating the user's beliefs). The bias may even be inherent to the source data (White and Hassan 2014;Kulshrestha et al 2017), and the search results may be biased even without any personalization and thus violate the openness norm (Bozdag and van den Hoven 2015) (e.g., when the number of followers of a religion is significantly lower compared to other religions and they have produced less online content). These problems can happen in almost any topic in which there are conflicting opinions, like many political, social, and even scientific topics.…”

Health Misinformation in Search and Social Media

“…• As prior research shows (White and Hassan, 2014), people have an uncontrolled bias towards believing that treatments are helpful, regardless of the ground-truth. (Chapter 5)…”

“…Related to the potential biases in medical web pages as well as among searchers, the vast amount of work conducted by White (2013), White and Hassan (2014), White (2014), White and Horvitz (2015) showed that searchers, as well as search engines, strongly favor positive information over negative information regardless of the truth. In his work, White looked at two different types of questions: medical queries with a yes/no question form (White, 2013;White and Hassan, 2014) as well as medical queries about the efficacy of medical treatments (White, 2014;White and Horvitz, 2015).…”

“…As the focus of this study is the behavior of users who post on social media health content of a questionable nature, we begin by compiling a set of purported cancer "cures" which have been shown by experimentation and medical professionals to be ineffective. Four of such dubious cancer treatments come from White and Hassan (2014), where authors judged and reached a consensus about the medical treatments' efficacy by reading the corresponding Cochrane Review (Cipriani et al, 2011;Higgins et al, 2008). Next, we collect nine rumor topics from the David Colquhoun (Professor of Pharmacology at University College London) blog 2 .…”

“…An example of an unhelpful treatment would be: Do insoles help back pain? Among the 249 medical treatments provided by White and Hassan (2014), we select five helpful treatments and five unhelpful treatments. Table 5.1 lists the ten medical treatments with their corresponding Cochrane ID.…”