The Criteria People Use in Relevance Decisions on Health Information: An Analysis of User Eye Movements When Browsing a Health Discussion Forum

Abstract

BackgroundPeople are increasingly accessing health-related social media sites, such as health discussion forums, to post and read user-generated health information. It is important to know what criteria people use when deciding the relevance of information found on health social media websites, in different situations.ObjectiveThe study attempted to identify the relevance criteria that people use when browsing a health discussion forum, in 3 types of use contexts: when seeking information for their own health issue, when seeking for other people’s health issue, and when browsing without a particular health issue in mind.MethodsA total of 58 study participants were self-assigned to 1 of the 3 use contexts or information needs and were asked to browse a health discussion forum, HealthBoards.com. In the analysis, browsing a discussion forum was divided into 2 stages: scanning a set of post surrogates (mainly post titles) in the summary result screen and reading a detailed post content (including comments by other users). An eye tracker system was used to capture participants’ eye movement behavior and the text they skim over and focus (ie, fixate) on during browsing. By analyzing the text that people’s eyes fixated on, the types of health information used in the relevance judgment were determined. Post-experiment interviews elicited participants’ comments on the relevance of the information and criteria used.ResultsIt was found that participants seeking health information for their own health issue focused significantly more on the poster’s symptoms, personal history of the disease, and description of the disease (P=.01, .001, and .02). Participants seeking for other people’s health issue focused significantly more on cause of disease, disease terminology, and description of treatments and procedures (P=.01, .01, and .02). In contrast, participants browsing with no particular issue in mind focused significantly more on general health topics, hot topics, and rare health issues (P=.01, .01, and .01).ConclusionUsers browsing for their own health issues used mainly case-based relevance criteria to relate the poster's health situation to their own. Participants seeking for others’ issues used mostly general knowledge–based criteria, whereas users with no particular issue in mind used general interest– and curiosity-based criteria.

Social science research data curation

“…In the summary search result screen of an information retrieval system, the user browses a list of document surrogates, comprising selected metadata fields, to make predictive relevance judgments-to identify documents that are likely to contain relevant information (Pian, Khoo and Chang 2016). For these selected records, the user then reads the detailed document content (if the full text is available) and makes evaluative relevance judgments of whether some information in the document content is relevant.…”

Automatic Classification of Users’ Health Information Need Context: Logistic Regression Analysis of Mouse-Click and Eye-Tracker Data

“…There are various types of health information available on the Internet from general medical terms to users’ experience of chronic diseases and drugs [ 4 ]. It takes time and effort for Internet users to find health information that is relevant to their situation while filtering out non-relevant information.…”

“…Previous studies have found that users seeking health information on the Internet can be categorized into the following three types of user health information need contexts: searching for the users’ own health issue, searching for someone else’s health issue (ie, for a family member or friend), and browsing with no particular health issue in mind [ 4 , 7 ]. When browsing for own health issue, users tend to make use of case-based relevance judgment to match the information with their own health condition (eg, personal history of disease and personal symptoms).…”

“…When browsing with no particular health issue in mind, users tend to focus on issues of general interest, current hot topics, and unusual diseases. Users in these three types of health information contexts make use of different criteria in assessing the relevance and usefulness of health information that they encounter on the Internet [ 4 ]. Understanding the relations between users’ searching or browsing patterns on health information websites and the types of user information need can help website and Web application developers to customize health information for particular categories of users and to increase the likelihood of relevance of the information [ 8 ].…”

“…Hence, mouse click patterns and, in the future, eye movement data are possible ways of identifying the category of user to achieve some customization of health information. Compared with user browsing patterns such as reading time and mouse clicks [ 9 - 14 ], eye movement data provide more insights on user’s relevance decision on whether health information is relevant [ 4 , 7 ].…”