Predicting searcher frustration

Abstract

When search engine users have trouble finding information, they may become frustrated, possibly resulting in a bad experience (even if they are ultimately successful). In a user study in which participants were given difficult information seeking tasks, half of all queries submitted resulted in some degree of self-reported frustration. A third of all successful tasks involved at least one instance of frustration. By modeling searcher frustration, search engines can predict the current state of user frustration and decide when to intervene with alternative search strategies to prevent the user from becoming more frustrated, giving up, or switching to another search engine. We present several models to predict frustration using features extracted from query logs and physical sensors. We are able to predict frustration with a mean average precision of 66% from the physical sensors, and 87% from the query log features.

Predicting web searcher satisfaction with existing community-based answers

“…We show a number of actual examples in Table 4. In the first example (E1), our method is able to correctly detect that the query is a little vague 7 , the query-question match is low 8 , and the answer is too simple to convince the searcher 9 . On the other hand, in the second example (E2), the query is quite clear and matches the question well, and the answer provides helpful advice to the searcher.…”

“…Searcher satisfaction in web search was addressed in [15,8,13], which utilized query log information for the task, such as relevance measures, as well as user behavior during the search session, including mouse clicks and time spent between user actions. What makes this task such a challenging problem is the large diversity in user goals [26], with a different definition of satisfaction for each, which requires developing unique satisfaction prediction models for the respective information needs.…”

Using Contextual Information to Understand Searching and Browsing Behavior

“…Previous research proposed different methods for identifying successful sessions: Hassan et al [97] used a Markov model to predict success at the end of the task; Ageev et al [9] exploited an expertise-dependent difference in search behavior by using a Conditional Random Fields model to predict a search success -authors used a game-like strategy for collecting annotated data by asking participants to find answers to non-trivial questions using web search. On the other hand, situations when users are frustrated have also been studied: Feild et al [75] proposed a method for understanding user frustration. Hassan et al [98] and Hassan Awadallah et al [99] have found that high similarity of queries is an indicator of an unsuccessful task.…”

“…On the other hand, situations when users are frustrated have also been studied. Feild et al [75] proposed a method for understanding user frustration. Hassan et al [98] and Hassan Awadallah et al [99] have found that high similarity of successive queries is an indicator of an unsuccessful task.…”

Towards User-Centric Cross-Site Personalisation

“…However this approach cannot assist the user in online tasks which require cross-site browsing, such as exploring product related information across enterprise and user generated Websites. The result of this fragmented browsing experience can increase user frustration through repetitive query usage within the different Websites [2]. To unify the fragmented browsing experience both the need of the user (freely browsing across the web) and the need of the content provider (encouraging the user to stay on the Website as long as possible) needs to be addressed.…”