Users often conduct exploratory searches on the Web to solve complex problems, but how for evaluating the effectiveness of a search, especially in a user-centered approach, remains an open question. We propose a user-centered method to evaluate the effectiveness of an exploratory search by focusing on the change in the users' mental representations of a topic during their exploratory search on the Web. This was done by comparing the concept maps depicted before and after each users searching. Thirty-five undergraduate students recruited from various departments and universities participated in our experiment. The participants were instructed to search for and gather Web pages for the task of writing a term paper on two given topics, politics and media, while working in either of two scenarios: "Selective scenario" (identifying only ten Web pages as important) and "As-Many-As scenario" (collecting as many Web pages as possible). The participants were divided into two groups: one group searched for both topics in the Selective scenario, and the other searched for them in the As-Many-As scenario. They composed their concept maps before and after searching. We defined the following measures to show the differences between the maps before and after a search to analyze the concept maps made by the participants: common, new, and lost map components like nodes, links, and link labels. We used these measures to compare the results from different topics, scenarios, and browser types. The results showed that the concept maps greatly changed after searching and that there were more common nodes and link labels in the Selective scenario than in the As-Many-As scenario. A comparative analysis of the concept maps between the pre-and post-search maps indicated that the users significantly changed their knowledge structure of a topic by completing the exploratory search task.
In this study, we investigate the changes in users' mental representations of a topic during their exploratory search on the Web. We use concept maps to compare the users' knowledge before and after a search. Participants are required to gather information on the Web as preparation for a regular feature of a magazine on environment and trip topics. They are divided into two scenario groups: divergent and convergent. The divergent scenario requires the gathering of Web pages for an entire series of articles in a regular feature while the convergent scenario requires the gathering of Web pages for a single article of a regular feature. A comparative analysis between the pre-and postsearch concept maps indicate that the participants in the divergent scenario change their knowledge widely while the participants in the convergent scenario change their knowledge deeply. The results also show that there were more nodes, links, and link labels in the trip topic than in the environmental topic.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.