News stories relevance effects on eye-movements

Abstract

Relevance is a fundamental concept in information retrieval. We consider relevance from the user's perspective and ask if the degree of relevance can be inferred from eye-tracking data and if it is related to the cognitive effort involved in relevance judgments. To this end we conducted a study, in which participants were asked to find information in screen-long text documents containing news stories. Each participant responded to fourteen trials consisting of an information question followed by three documents each at a different level of relevance (irrelevant, partially relevant, and relevant). The results indicate that relevant documents tended to be continuously read, while irrelevant documents tended to be scanned. In most cases, cognitive effort inferred from eye-tracking data was highest for partially relevant documents and lowest for irrelevant documents.

Factuality Checking in News Headlines with Eye Tracking

“…Our findings complement prior findings that news posts from credible sources receive more gaze attention [13] and that false news tend to be read more quickly than accurate news [6]. However, none of the above studies is done on headlines, and, to our knowledge, we present the first factuality inference model to be trained exclusively on eye-tracked data.…”

“…Eye tracking has long been used in IR to infer relevance [1,3,4,7,8,11] and to improve user understanding, for instance that adding information to search engine snippets significantly improves performance for informational tasks but degrades performance for navigational tasks [5]; that users with higher change in knowledge differ significantly in terms of the number and duration of fixations compared to users with lower knowledge-change [2]; and that relevant documents tend to be continuously read, while irrelevant documents tend to be scanned [6]. In most cases, cognitive effort inferred from eye-tracking data is highest for (at least) partially relevant documents and lowest for irrelevant documents.…”

Review and Analysis of Machine Learning and Soft Computing Approaches for User Modeling

“…Gwizdka and Jacek [17] contributed their work in personalization for news stories relevance from user's perspective. They inferred relevance from eye-tracking data and observed if it is related to the cognitive effort involved in relevance judgments.…”

Relevance Prediction from Eye-movements Using Semi-interpretable Convolutional Neural Networks

“…Buscher et al [6] used reading-to-skimming ratio to infer when participants were reading relevant text. Over a group of studies, Gwizdka et al [19][20][21][22] reported that reading speed, number of fixations on words, count and length of reading sequences, count and percentage of words fixated upon, durations of reading and scanning, and distance covered by scanning proved to be good indicators of perceived-relevance for textual documents.…”