Context effect on query formulation and subjective relevance in health searches

Abstract

It is recognized by the Information Retrieval community that context affects the retrieval process. Query formulation and relevance assessment are stages where the user role is central. The first determines what the system will search for and the second is frequently used to evaluate how the system behaved. With a large human involvement, these stages are expected to be largely influenced by user and task characteristics. To analyze the influence of these context features on the specified stages of health information retrieval, we conducted a user study in which we collected user features through two questionnaires. User characteristics include features like age, gender, web search experience, health search experience and familiarity with the medical topic. Task features include the medical specialty, the question type, the task's clarity and the task's easiness. Besides user and task features, the relevance assessment analysis also covered features related to the query and document. We found many variables do indeed affect query formulation and relevance judgment. Some of our results question evaluations using test collections and ask for evaluation models that incorporate other kind of success measures.

Using the Characteristics of Documents, Users and Tasks to Predict the Situational Relevance of Health Web Documents

“…The clarity of the tasks was also found to contribute positively to relevance, while the familiarity of users with the tasks showed negative contribution. More experienced users might be more demanding, what is inline with previous findings (Lopes et al 2010, Saracevic 1996.…”

“…Further development of search technologies for consumer health search considers self-diagnosis information needs and needs related to treatment and management of health conditions (Zuccon et al 2016). The relevance assessments were shown to be influenced by user, task, query and document characteristics (e.g., age, gender, health search experience, medical specialty, task clarity) (Lopes et al 2010). A previous study showed that user and task characteristics are also good descriptors and possible predictors of relevance (Oroszlányová et al 2015).…”

“…A previous study showed that user and task characteristics are also good descriptors and possible predictors of relevance (Oroszlányová et al 2015). In the present work, we want to predict the relevance of a document to a user, with the help of the available features (Lopes et al 2010).…”

Effects of Terminology on Health Queries: An Analysis by User’s Health Literacy and Topic Familiarity

“…Only one study was found that considered users' topic familiarity in health IR (Lopes & Ribeiro, 2010). Its authors studied the impact of several context features on query formulation and relevance assessment in health searches and concluded that, in more familiar tasks, users employ medico-scientific terminology more often and formulate longer queries.…”

The presentation of health-related search results and its impact on negative emotional outcomes

“…Research has demonstrated that individuals who have a better understanding of online health information are more likely to use the Internet, over a doctor, as their primary source [18]. Additionally, individuals who have engaged in frequent health searches are less likely to judge search results for health topics as relevant to their initial queries, suggesting that they may be more critical of the results retrieved [19]. To explore the role of online health experience on individuals' responses to health-related search results, the following research question is posed: RQ1: Does the extent of experience with online health information moderate the relationships between placement and severity and frequency and susceptibility?…”