Consumer Health Search on the Web: Study of Web Page Understandability and Its Integration in Ranking Algorithms (Preprint)

Abstract

Background: Understandability plays a key role in ensuring that people accessing health information are capable of gaining insights that can assist them with their health concerns and choices. The access to unclear or misleading information has been shown to negatively impact on the health decisions of the general public. Objective:We investigated methods to estimate the understandability of health Web pages and used these to improve the retrieval of information for people seeking health advice on the Web. Methods:Our investigation considered methods to automatically estimate the understandability of health information in Web pages, and it provided a thorough evaluation of these methods using human assessments as well as an analysis of preprocessing factors affecting understandability estimations, and associated pitfalls. Furthermore, lessons learnt for estimating Web page understandability were applied to the construction of retrieval methods with specific attention to retrieving information understandable by the general public. Results:We found that machine learning techniques were more suitable to estimate health Web page understandability than traditional readability formulae, which are often used as guidelines and benchmarking by health information providers on the Web (larger difference found for Pearson correlation of .602 using Gradient Boosting regressor compared to .438 using SMOG Index with CLEF 2015 collection). Learning to rank effectively exploited these estimates to provide the general public with more understandable search results ( H RBP ¿ reached 29.20, 22% higher than a BM25 baseline and 13% higher than the best system at CLEF 2016, both P≤ .001 ).