Large-scale analysis of individual and task differences in search result page examination strategies

Abstract

Understanding the impact of individual and task differences on search result page examination strategies is important in developing improved search engines. Characterizing these effects using query and click data alone is common but insufficient since they provide an incomplete picture of result examination behavior. Cursor-or gaze-tracking studies reveal richer interaction patterns but are often done in small-scale laboratory settings. In this paper we leverage large-scale rich behavioral log data in a naturalistic setting. We examine queries, clicks, cursor movements, scrolling, and text highlighting for millions of queries on the Bing commercial search engine to better understand the impact of user, task, and user-task interactions on user behavior on search result pages (SERPs). By clustering users based on cursor features, we identify individual, task, and user-task differences in how users examine results which are similar to those observed in small-scale studies. Our findings have implications for developing search support for behaviorally-similar searcher cohorts, modeling search behavior, and designing search systems that leverage implicit feedback.

Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts

“…Other work from Huang, Buscher and colleagues [17,16,4] compare eye and mouse tracking results from lab studies to large-scale logs data from a search engine, deployed both internally [17] and on an external sample of users [16,4]. This work demonstrated that mouse-based data can be used to evaluate search result relevance, distinguish cases of "good" (user need-satisfying) and "bad" abandonment on web pages, and identify clusters of distinct search task strategies.…”

“…We identify mouse measures that are most correlated with eye gaze. 4. We develop models that predict users' eye gaze reasonably well from their mouse activity (67% accuracy in predicting the fixated result element, with an error of upto one element).…”

A Context-aware Time Model for Web Search

“…Besides using context-independent estimates of times between consecutive clicks as features for ranking, the predictions of context-aware time models can be used in a range of other applications that use the average or predicted times between user actions. These applications include prediction of click satisfaction [30], result usefulness [32], search task difficulty [3,33], search goal success [20,21], urgent information needs [35], struggling vs. exploring behavior [22,36], positive vs. negative abandonment [41] and clustering users based on their SERP examination strategies [7].…”

“…and the average time-to-first-click observed for a user to cluster users based on their SERP examination strategies [7]. Time-between-clicks is the time between two consecutive clicks on search engine results (undefined for the last click on search results).…”

“…Existing work uses it as a feature for ranking [1,2,8] and in many tasks related to user satisfaction [3, 20-22, 30, 32, 33, 36]. Times from a submission of a query to(i) the first click [7,10,20,39], (ii) the first satisfied click [39], and (iii) the last click [10] are used as features to predict user satisfaction with the clicked result [20], compare two versions of a search engine [10,39] and to cluster users based on their SERP examination strategies [7]. Song et al [41] use the time since a user issued an abandoned query (i.e., a query for which there were no interactions with the search results) to the next query submission for classifying the abandoned query into positively abandoned and negatively abandoned.…”