Characterizing multi-click search behavior and the risks and opportunities of changing results during use

Abstract

Although searchers often click on more than one result following a query, little is known about how they interact with search results after their first click. Using large scale query log analysis, we characterize what people do when they return to a result page after having visited an initial result. We find that the initial click provides insight into the searcher's subsequent behavior, with short initial dwell times suggesting more future interaction and later clicks occurring close in rank to the first. Although users think of a search result list as static, when people return to a result list following a click there is the opportunity for the list to change, potentially providing additional relevant content. Such change, however, can be confusing, leading to increased abandonment and slower subsequent clicks. We explore the risks and opportunities of changing search results during use, observing, for example, that when results change above a user's initial click that user is less likely to find new content, whereas changes below correlate with increased subsequent interaction. Our results can be used to improve people's search experience during the course of a single query by seamlessly providing new, more relevant content as the user interacts with a search result page, helping them find what they are looking for without having to issue a new query.

Evaluation in Contextual Information Retrieval

“…The related nding clearly argues against the independence hypothesis. Recent studies on anchoring and adjustment in relevance estimation [88,126] prove that the human annotators are likely to assign di erent relevance labels to a document, depending on the quality of the last document they had judged for the same query. Regarding consistency, it is well known today that TREC and CLEF style experiments are generally based on expert assessments seen as objective, while real-life IR settings are based on real users for whom assessments are seen as subjective [19,60] and several contextual factors a ect the users when judging document relevance.…”