BackgroundTo earn HONcode certification, a website must conform to the 8 principles of the HONcode of Conduct In the current manual process of certification, a HONcode expert assesses the candidate website using precise guidelines for each principle. In the scope of the European project KHRESMOI, the Health on the Net (HON) Foundation has developed an automated system to assist in detecting a website’s HONcode conformity. Automated assistance in conducting HONcode reviews can expedite the current time-consuming tasks of HONcode certification and ongoing surveillance. Additionally, an automated tool used as a plugin to a general search engine might help to detect health websites that respect HONcode principles but have not yet been certified.ObjectiveThe goal of this study was to determine whether the automated system is capable of performing as good as human experts for the task of identifying HONcode principles on health websites.MethodsUsing manual evaluation by HONcode senior experts as a baseline, this study compared the capability of the automated HONcode detection system to that of the HONcode senior experts. A set of 27 health-related websites were manually assessed for compliance to each of the 8 HONcode principles by senior HONcode experts. The same set of websites were processed by the automated system for HONcode compliance detection based on supervised machine learning. The results obtained by these two methods were then compared.ResultsFor the privacy criterion, the automated system obtained the same results as the human expert for 17 of 27 sites (14 true positives and 3 true negatives) without noise (0 false positives). The remaining 10 false negative instances for the privacy criterion represented tolerable behavior because it is important that all automatically detected principle conformities are accurate (ie, specificity [100%] is preferred over sensitivity [58%] for the privacy criterion). In addition, the automated system had precision of at least 75%, with a recall of more than 50% for contact details (100% precision, 69% recall), authority (85% precision, 52% recall), and reference (75% precision, 56% recall). The results also revealed issues for some criteria such as date. Changing the “document” definition (ie, using the sentence instead of whole document as a unit of classification) within the automated system resolved some but not all of them.ConclusionsStudy results indicate concordance between automated and expert manual compliance detection for authority, privacy, reference, and contact details. Results also indicate that using the same general parameters for automated detection of each criterion produces suboptimal results. Future work to configure optimal system parameters for each HONcode principle would improve results. The potential utility of integrating automated detection of HONcode conformity into future search engines is also discussed.
IntroductionThe explosive growth of the World Wide Web has made it more and more urgent to monitor and improve the quality of the information circulating through the Internet. But assessing medical and health information on the Web is a difficult challenge. In recent years, some organisations have been working on this issue. The Health On the Net Foundation (HON), for its part, administers an eight-point Code of Conduct called the HONcode, initiated in 1996. This solution is different from the others. The HONcode does not intend to rate or assess the quality of information provided by a Web site. This article gives a general presentation of the HONcode and its status in 1999, three years after it was launched.MethodsIt defines a set of voluntary rules designed to help a Web developer site practice responsible and to make sure a reader always knows the source and the purpose of the information he or she is reading. These guidelines encourage the authority, complimentarity, confidentiality, proper attribution, justifiability and validity of the medical advice and information provided. Furthermore, sites that subscribe to the HONcode commit themselves to providing transparent information on site sponsorship and clearly separating advertising and editorial content.ResultsIn 1998, the number of external links to the HONcode principles page grew by about 250% (e.g., from 2 to 5) . The rate of increase from January through April, 1999, suggests 300% this year. The growth in the number of links to the HONcode matches that of links to of the entire Web site, and is slightly higher than growth in links to HON's MedHunt search engine. Statistical analysis shows that approximately 50% of Web sites linked to the HONcode principles have a commercial domain name, 16% are from European countries, 15% are from non-profit organisations, and 7% are educational sites.DiscussionThis evolution shows a real need of guidelines for the developers of medical and health Web sites and their users. This conclusion is reinforced by the result of an international survey which HON conducted on the Internet in March and April 1999. The HONcode's scope of application is currently limited to publishing policy and ethical aspects. The HONcode does not address issues of quality and the relevance of the health and medical content of a Web page -- and this is a crucial issue for the further healthy development of the Internet in the medical domain. One way to assess and provide reliable medical and health content could be to review medical and health-related Web content by peers. Building on the system adopted by the scientific community decades ago for the paper medium, this Electronic peer review method could cover content as well as ethical and publishing aspects while retaining use of the HONcode display icon on registered sites as an additional reference.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.