Judgment of information quality and cognitive authority in the Web

Abstract

In the Web, making judgments of information quality and authority is a difficult task for most users because overall, there is no quality control mechanism. This study examines the problem of the judgment of information quality and cognitive authority by observing people's searching behavior in the Web. Its purpose is to understand the various factors that influence people's judgment of quality and authority in the Web, and the effects of those judgments on selection behaviors. Fifteen scholars from diverse disciplines participated, and data were collected combining verbal protocols during the searches, search logs, and post-search interviews. It was found that the subjects made two distinct kinds of judgment: predictive judgment and evaluative judgment. The factors influencing each judgment of quality and authority were identified in terms of characteristics of information objects, characteristics of sources, knowledge, situation, ranking in search output, and general assumption. Implications for Web design which will effectively support people's judgments of quality and authority are also discussed. IntroductionOne of the advantages of searching in the Web is its grant of access to a great amount and a wide variety of information. As a result, however, people need some ways to reduce the large amount of information in order to select the information that they want. In traditional information retrieval, this problem has long been discussed within the context of "topical relevance"; that is, in terms of whether the topic of the query matches the topic of a document. However, a substantial number of empirical studies (e.g. Barry, 1994;Cool, Belkin, Frieder, & Kantor, 1993;Park, 1993;Schamber, 1991;Spink & Greisdorf, 2001;Wang & Soergel, 1999) have revealed that people use much more diverse criteria than mere topicality to make relevance judgments in the traditional information retrieval environment. This study will Rieh 2 take these findings a step further by focusing on two factors which appear consistently across the previous studies: quality and authority. These two factors were chosen because it is believed that they may be more important relevance criteria than any other criteria identified in the previous studies, especially in a large uncontrolled environment, such as the Web.The concepts of quality and authority are not new. On the one hand, a number of studies of relevance criteria, particularly in the 1990s, identified various aspects of both concepts including "goodness" (Cool et al.), "usefulness" (Cool et al.), "accuracy/validity" (Barry), "recency" (Barry; Wang & Soergel), "perceived quality (Park), "actual quality" (Wang & Soergel), "expected quality" (Wang & Soergel), "authority" (Cool et al.; Wang & Soergel), and "reliability" (Schamber). On the other hand, in recent years, the notions of quality and authority have been discussed with respect to evaluation criteria of Web pages by examining different approaches and implementations. Librarians and researchers in library and information science (e....

A quality evaluation methodology for health-related websites based on a 2-tuple fuzzy linguistic approach

“…This is an important and differential aspect because in other user-based approaches (Aladwani and Palvia 2002;Herrera-Viedma et al 2006;Herrera-Viedma and Peis 2003;Rieh 2002;Yang et al 2005) we find that the considered quality criteria are established by the experts using their experience or user-oriented quality evaluation models defined for other frameworks.…”

“…We consider that any website quality evaluation methodology should include the direct participation of the users to evaluate the quality of websites, because the concept of quality is typically consumer-dependent, and the consumer must be the ultimate judge of their quality. To evaluate websites focusing on the user-perceived quality is a difficult task that has seldom been studied, and there does not exist a Web quality framework as a reference point (Aladwani and Palvia 2002;Rieh 2002;Yang et al 2005). Furthermore, the aspects of quality to be considered and their relative importance vary according to the application domain and the goals to be achieved by the users.…”

“…This study was conducted with the participation of a representative sample of two specific classes of users that access regularly health-related websites: general visitors (chronic patients that do regular searches of websites) and expert visitors (doctors with a high level of knowledge in their medical fields). With this methodology, we increase the user participation in the quality evaluation process of websites with respect to other user-based approaches (Aladwani and Palvia 2002;Herrera-Viedma et al 2006;Herrera-Viedma and Peis 2003;Rieh 2002;Yang et al 2005), because it is based on the linguistic evaluation judgements provided by the users, but furthermore, the set of considered quality criteria is obtained from the real users' needs and expectations.…”

The Impact of User Knowledge on Web Search Satisfaction

“…Most importantly, the information (search result) is considered relevant when it matches the query entered during the search session (Rieh, 2002). This judgment is typical as most of the search systems provide this facility as a part of their services.…”

Modeling Patent Quality: A System for Large-scale Patentability Analysis using Text Mining

“…Alexander and Tate pointed out that manual lists of good Web pages can cover only a small fraction of the documents on the ever-growing Web [1]. Rieh compared the relationships between various types of characteristic factors of the Web sites and questionnaire results about their information quality and perceived authority [18]. In particular, medical researchers have paid close attention to the quality of medical information on the Internet, since users' misunderstanding based on incorrect information on the Web can potentially cause fatal medical accidents.…”