Combining eye movements and collaborative filtering for proactive information retrieval

Abstract

We study a new task, proactive information retrieval by combining implicit relevance feedback and collaborative filtering. We have constructed a controlled experimental setting, a prototype application, in which the users try to find interesting scientific articles by browsing their titles. Implicit feedback is inferred from eye movement signals, with discriminative hidden Markov models estimated from existing data in which explicit relevance feedback is available. Collaborative filtering is carried out using the User Rating Profile model, a state-of-the-art probabilistic latent variable model, computed using Markov Chain Monte Carlo techniques. For new document titles the prediction accuracy with eye movements, collaborative filtering, and their combination was significantly better than by chance. The best prediction accuracy still leaves room for improvement but shows that proactive information retrieval and combination of many sources of relevance feedback is feasible.

Emotion based Contextual Semantic Relevance Feedback in Multimedia Information Retrieval

“…We can have only guess and estimate the semantic by using emotional reactions of a user when s(he) has shown interest on multimedia contents during retrieval process. Semantic can be guessed and estimated by capturing and interpreting a stream of user's actions (browse and navigation, eye tracking, etc) during interaction with multimedia contents [7,5]. It is suggested that relevance feedback can be utilized for affective retrieval [8] but still require an efficient affective feedback system that can capture, register and interpret up to some extents spontaneous reactions of a user and can use it to identify user's current context amongst multiple contexts present in a user's search query.…”

“…Fig. 2, yields the final discriminating function and is used in training and recognizing the spoken emotional word.Several attempts have beenintroduced in thepast to build user's profileand learning user interest from implicit feedback [35,5].…”

“…Based on such feedback information, the system updates a distance metric among the low-level features to better reflect high-level conceptual semanticdistances and modify the parametric space, feature space, semantic space, orclassification space to identify the relevant and irrelevant information. Various implicit [2,3] and explicit [4] relevance feedbacks and combination of implicit relevance feedback with explicit relevance feedback [5] methods have been proposed in recent years to deal with and to remove ambiguity of user's information needs, but not sufficiently as may be desired.…”

Learning to rank images from eye movements

“…Eye movements however can also be treated as an implicit feedback when the user is not consciously trying to influence the interface by where they focus their attention. Eye movements as implicit feedback has recently been considered in the text retrieval setting [11,4,1]. To the best of our knowledge however, at the time of writing, only [10,8] used eye movements for image retrieval.…”

A performance comparison of eye tracking and mouse interfaces in a target image identification task

“…More robust methods of interpreting the data are needed. There has been some recent work on document retrieval in which eye tracking data has been used to refine the accuracy of relevance predictions [13].…”