Source evaluation of domain experts and novices during Web search

Abstract

Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub-skill in this process. But many students have not yet optimally developed this skill. On the basis of verbal reports, eye-tracking data, and navigation logs this study investigated how novices in the domain of psychology evaluate Internet sources as compared to domain experts. In addition, two different verbal reporting techniques, namely thinking aloud and cued retrospective reporting, were compared in order to examine students' evaluation behavior. Results revealed that domain expertise has an impact on individuals' evaluation behavior during Web search, such that domain experts showed a more sophisticated use of evaluation criteria to judge the reliability of sources and information and selected more reliable information than domain novices. Furthermore, the different verbal reporting techniques did not lead to different conclusions on criteria use in relation to domain expertise, although in general more utterances concerning evaluation of sources and information were expressed during cued retrospective reporting.

“I'm Always Kind of Double‐Checking”: Exploring the Information‐Seeking Identities of Expert Generalists

“…Goldman and Scardamalia () identified “multiple document comprehension” as “a prerequisite of digital literacy” (p. 255), and research in this area often has involved presenting participants with multiple sources of information or a list of results from a (real or simulated) search engine. Research methods and data include think‐aloud protocols (e.g., Goldman, Braasch, Wiley, Graesser, & Brodowinska, ), eye‐tracking data (e.g., Brand‐Gruwel, Kammerer, van Meeuwen, & van Gog, ; Gerjets, Kammerer, & Werner, ; Wiley et al., ), assessments of task performance such as writing a report or letter (e.g., Bråten, McCrudden, Lund, Brante, & Strømsø, ; Goldman et al., ), or making recommendation (e.g., Gerjets et al., ). In analyzing data, previous researchers have evaluated task performance in terms of the amount of factually accurate information.…”

“…() established a connection between students’ learning from information online and their ability to successfully evaluate the reliability of that information. Learning online is often connected to the complex process of solving an information problem (Brand‐Gruwel et al., ), which involves multiple skills, including evaluating the relevance and credibility of sources (for a description of the multiple‐document task‐based relevance assessment and content extraction model, upon which some research in this area is based, see Rouet & Britt, ). In a recent study that aimed to understand what weight high school students gave to content relevance and author expertise, Bråten et al.…”

“…Their research has found that historians, those whose disciplinary literacy skills were hypothesized to be the most compatible with online literacy demands, were often fooled by deceptive websites. Previous research involving experts and information evaluation has suggested that experts are superior to novices when the information is connected to their area of expertise (e.g., Brand‐Gruwel et al., ; Lucassen & Schraagen, ); in other words, background knowledge can be leveraged when evaluating online information. Yet, contemporary online literacy research often has involved investigating topics about which the information seeker has little or no background knowledge (e.g., consumer decisions, personal health needs, civic issues; see Kohnen & Saul, ).…”

Credibility Dimensions for Islamic Information in Social Media

“…A link to other sources refers to a website address delivered in the message posted for the reader to acquire additional information [56]. This is to provide better coverage and comprehensiveness of the source of information [9]. Hence, trustworthiness is undoubtedly a critical factor in credibility evaluation.…”

“…The literature indicates that Internet users lack the training and skills necessary to evaluate the integrity of a website with accurate information [8], [9]. In recent years, there has been an increasing interest in the exploration of information credibility by focusing on the information published on websites and blogs [10].…”