Captions and biases in diagnostic search

Abstract

People frequently turn to the Web with the goal of diagnosing medical symptoms. Studies have shown that diagnostic search can often lead to anxiety about the possibility that symptoms are explained by the presence of rare, serious medical disorders, rather than far more common benign syndromes. We study the influence of the appearance of potentially-alarming content, such as severe illnesses or serious treatment options associated with the queried for symptoms, in captions comprising titles, snippets, and URLs. We explore whether users are drawn to results with potentially-alarming caption content, and if so, the implications of such attraction for the design of search engines. We specifically study the influence of the content of search result captions shown in response to symptom searches on search-result click-through behavior. We show that users are significantly more likely to examine and click on captions containing potentially-alarming medical terminology such as "heart attack" or "medical emergency" independent of result rank position and well-known positional biases in users' search examination behaviors. The findings provide insights about the possible effects of displaying implicit correlates of searchers' goals in search-result captions, such as unexpressed concerns and fears. As an illustration of the potential utility of these results, we developed and evaluated an enhanced click prediction model that incorporates potentially-alarming caption features and show that it significantly outperforms models that ignore caption content. Beyond providing additional understanding of the effects of Web content on medical concerns, the methods and findings have implications for search engine design. As part of our discussion on the implications of this research, we propose procedures for generating more representative captions that may be less likely to cause alarm, as well as methods for learning to more appropriately rank search results from logged search behavior, for examples, by also considering the presence of potentially-alarming content in the captions that motivate observed clicks and down-weighting clicks seemingly driven by searchers' health anxieties.

Investigating the role of semantic priming in query expression: A framework and two experiments

“…Searchers focus on the top two summaries, where click‐through is more likely, and they give little attention to summaries lower on the list (Cutrell & Guan, ; Dumais, Buscher, & Cutrell, ; Guan & Cutrell, ; Joachims et al., ). Other characteristics also affect interaction, including the length of summary text, the appearance of query terms in summaries, and domain‐specific semantic characteristic (Cutrell & Guan, ; White & Horvitz, ). Studies of search over unranked lists and webpages find that searchers adapt their scanning and assessment patterns to match task demands (Simola, Salojärvi, & Kojo, ), the visual characteristics of the list (Tarling & Brumby, ), and relevance (Brumby & Howes, ; Gwizdka, ).…”

Belief dynamics in web search

“…Topic‐related biases may also apply in personalized search settings where the search engine may choose to exclude certain information in light of a user's recent search interests (Pariser, ). By considering the cognitive factors affecting result selection decisions as well as the click evidence itself (e.g., whether clicks are associated with biased beliefs or escalatory content [White & Horvitz, ]), more accurate ranking models can be developed. Overall, methods for better understanding and detecting bias‐related search interactions are required, for example, for question queries, clicks on captions with supporting ( yes ‐oriented) evidence could be down‐weighted because we observe those to occur significantly more frequently than expected (59%) given base rates (50%) and those clicks may be driven by factors beyond relevance or accuracy.…”

Interactions between health searchers and search engines

“…Note that during the time period analyzed in this study, the search engine only returned eight results on a large number of SERPs so we disregarded possible further results. It has been shown that users rarely click below position 8, including for health queries [36]. The classifier is based on page features from the URL and HTML content similar to those shown in Table 2.…”

“…The engine, registering the words "severe" and "explanations" as well as the phrase "brain tumor" present in the user's search history might compile a search engine result page (SERP) that is biased towards serious conditions. The user, viewing the SERP through the lens of their current health anxiety, may be attracted towards serious conditions in captions [36] and hence select a concerning page, heightening their anxiety further.…”