Investigating exploratory search activities based on the stratagem level in digital libraries

Abstract

In this paper we present the results of a user study on exploratory search activities in a social science digital library. We conducted a user study with 32 participants with a social sciences background -16 postdoctoral researchers and 16 students -who were asked to solve a task on searching related work to a given topic. The exploratory search task was performed in a 10-minutes time slot. The use of certain search activities is measured and compared to gaze data recorded with an eye tracking device. We use a novel tree graph representation to visualise the users' search patterns and introduce a way to combine multiple search session trees. The tree graph representation is capable to create one single tree for multiple users and to identify common search patterns. In addition, the information behaviour of students and postdoctoral researchers is being compared. The results show that search activities on the stratagem level are frequently utilised by both user groups. The most heavily used search activities were keyword search, followed by browsing through references and citations, and author searching. The eye tracking results showed an intense examination of documents metadata, especially on the level of citations and references. When comparing the group of students and postdoctoral researchers we found significant differences regarding gaze data on the area of the journal name of the seed document. In general, we found a tendency of the postdoctoral researchers to examine the metadata records more intensively with regards to dwell time and the number of fixations. By creating combined session trees and deriving subtrees from those, we were able to identify common patterns like economic (explorative) and exhaustive (navigational) behaviour. Our results show that participants utilised multiple search strategies starting from the seed document, which means, that they examined different paths to find related publications.

Analysis of search stratagem utilisation

“…In [10], the authors study exploratory search tasks that require users to investigate obtained results in DLs. They focused on early stages: "starting", "chaining", and "browsing".…”

“…The goal of this paper 2 is to explore the specific stratagems "Footnote Chasing", "Citation Searching", "Keyword Searching", "Author Searching" and "Journal Run" which are often utilised as exploratory search functionalities in DLs [12,10]. The relationship between Information Retrieval and Scientometrics has been discussed in a recent special issue in Scientometrics [32].…”

“…The gain is computed as the percentage of increase between the precision-after and the precision-before 10. The baseline is a set of 100 random sessions that do not include any of the stratagems.…”

Introduction to the special issue on bibliometric-enhanced information retrieval and natural language processing for digital libraries (BIRNDL)

“…2 This special issue resulting from the BIRNDL workshop includes 14 papers: four extended papers presented at the first BIRNDL workshop and the BIR workshop at ECIR 2016 [2,8,14,18], three extended system reports of the CL-SciSumm Shared Task 2016 [1,13,16] and one overview paper [11] and six original research papers submitted via the open call for papers [6,7,9,10,12,17]. guage Processing publications" [14] to detect extrinsic instances of self-reuse, self-plagiarism, reuse and plagiarism in NLP and speech processing articles.…”

“…Task-oriented filtering also turns out to be cost-saving, where cost is defined as the number of articles users have to view before reaching a certain gain. -In "Investigating exploratory search activities based on the stratagem level in digital libraries" [6], Carevic et al present a user study on exploratory search for a given search task in the social science digital library sowiport. The users, 32 participants in total (16 postdocs and 16 students), were asked to perform a 10 minutes exploratory search task "finding and collecting similar documents" starting from a predefined relevant seed document.…”