Robust models of mouse movement on dynamic web search results pages

Dı́az, Fernando; White, Ryen W.; Buscher, Georg; Liebling, Dan

doi:10.1145/2505515.2505717

Cited by 52 publications

(33 citation statements)

References 40 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The utility of mouse tracking has been demonstrated for a number of applications, such as understanding search result page examination [18,19,20], improving results ranking [7,38], and performing relevance predictions [14,16]. Although the importance of mouse tracking data in web search is now evident, very little is known about within-content engagement.…”

Section: Introductionmentioning

confidence: 99%

Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures

Arapakis

Lalmas

Valkanas

2014

Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

The availability of large volumes of interaction data and scalable data mining techniques have made possible to study the online behaviour for millions of Web users. Part of the efforts have focused on understanding how users interact and engage with web content. However, the measurement of within-content engagement remains a difficult and unsolved task. This is because of the lack of standardised, wellvalidated methods for measuring engagement, especially in an online context. To address this gap, we perform a controlled user study where we observe how users respond to online news in the presence or lack of interest. We collect mouse tracking data, which are known to correlate with visual attention, and examine how cursor behaviour can inform user engagement measures. The proposed method does not use any pre-determined concepts to characterise the cursor patterns. We, rather, follow an unsupervised approach and use a large set of features engineered from our data to extract the cursor patterns. Our findings support the connection between gaze and cursor behaviour but also, and more importantly, reveal other dependencies, such as the correlation between cursor activity and experienced affect. Finally, we demonstrate the value of our method by predicting the outcome of online news reading experiences.

show abstract

Section: Introductionmentioning

confidence: 99%

Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures

Arapakis

Lalmas

Valkanas

2014

Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

show abstract

“…Some work distinguishes different types of clicks: first click [8,39], last click [8], long dwell time click [8], satisfied click [39] and only click [8]. Next to clicks, some recent work considers mouse cursor movements on SERPs [16,24,25]. Behavioral signals based on times between user actions.…”

Section: Behavioral Informationmentioning

confidence: 99%

A Context-aware Time Model for Web Search

Borisov

Markov

Rijke

et al. 2016

Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

A context-aware time model for web search Borisov, A.; Markov, I.; de Rijke, M.; Serdyukov, P. General rightsIt is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulationsIf you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: http://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. ABSTRACTIn web search, information about times between user actions has been shown to be a good indicator of users' satisfaction with the search results. Existing work uses the mean values of the observed times, or fits probability distributions to the observed times. This implies a context-independence assumption that the time elapsed between a pair of user actions does not depend on the context, in which the first action takes place. We validate this assumption using logs of a commercial web search engine and discover that it does not always hold. For between 37% to 80% of queryresult pairs, depending on the number of observations, the distributions of click dwell times have statistically significant differences in query sessions for which a given result (i) is the first item to be clicked and (ii) is not the first. To account for this context bias effect, we propose a context-aware time model (CATM). The CATM allows us(i) to predict times between user actions in contexts, in which these actions were not observed, and (ii) to compute contextindependent estimates of the times by predicting them in predefined contexts. Our experimental results show that the CATM provides better means than existing methods to predict and interpret times between user actions.

show abstract

“…Implicit feedback consists for instance of query reformulations [71], mouse clicks [95], mouse movements [48,66,67,73,201], measurements of dwell-time [206]-the time users spend on a website-, or even the time a search result [115]. The advantage of implicit feedback over explicit feedback is that it is available in abundance.…”

Section: Interpreting User Interactionsmentioning

confidence: 99%

“…If a user never looked at a search result, their lack of engagement on this result cannot be indicative of low relevance. A number of studies have shown that mouse movement can be an indicator of user examination of search results, and of specific sections within search results [48,66,67,73,201]. Similarly, in a mobile setting recording how long each part of the screen is visible can be considered an indicator of relevance [115].…”

Section: A/b Testingmentioning

confidence: 99%

Search Engines that Learn from Their Users

Schuth

2016

SIGIR Forum

View full text Add to dashboard Cite

More than half the world's population uses web search engines, resulting in over half a billion queries every single day. For many people, web search engines such as Baidu, Bing, Google, and Yandex are among the first resources they go to when a question arises. Moreover, for many search engines have become the most trusted route to information, more so even than traditional media such as newspapers, news websites or news channels on television. What web search engines present people with greatly influences what they believe to be true and consequently it influences their thoughts, opinions, decisions, and the actions they take. With this in mind two things are important, from an information retrieval research perspective. First, it is important to understand how well search engines (rankers) perform and secondly this knowledge should be used to improve them. This thesis is about these two topics: evaluation of search engines and learning search engines. In the first part of this thesis we investigate how user interactions with search engines can be used to evaluate search engines. In particular, we introduce a new online evaluation paradigm called multileaving that extends upon interleaving. With multileaving, many rankers can be compared at once by combining document lists from these rankers into a single result list and attributing user interactions with this list to the rankers. Then we investigate the relation between A/B testing and interleaved comparison methods. Both studies lead to much higher sensitivity of the evaluation methods, meaning that fewer user interactions are required to arrive at reliable conclusions. This has the important implication that fewer users need to be exposed to the results from possibly inferior search engines. In the second part of this thesis we turn to online learning to rank. We learn from the evaluation methods introduced and extended upon in the first part. We learn the parameters of base rankers based on user interactions. Then we use the multileaving methods as feedback in our learning method, leading to much faster convergence than existing methods. Again, the important implication is that fewer users need to be exposed to possibly inferior search engines as they adapt more quickly to changes in user preferences. The last part of this thesis is of a different nature than the earlier two parts. As opposed to the earlier chapters, we no longer study algorithms. Progress in information retrieval research has always been driven by a combination of algorithms, shared resources, and evaluation. In the last part we focus on the latter two. We introduce a new shared resource and a new evaluation paradigm. Firstly, we propose Lerot. Lerot is an online evaluation framework that allows us to simulate users interacting with a search engine. Our implementation has been released as open source software and is currently being used by researchers around the world. Secondly we introduce OpenSearch, a new evaluation paradigm involving real users of real search engines. We describe an implementation of this paradigm that has already been widely adopted by the research community through challenges at CLEF and TREC.

show abstract

Robust models of mouse movement on dynamic web search results pages

Cited by 52 publications

References 40 publications

Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures

Understanding Within-Content Engagement through Pattern Analysis of Mouse Gestures

A Context-aware Time Model for Web Search

Search Engines that Learn from Their Users

Contact Info

Product

Resources

About