“Pretty Rad”: Explorations in User Satisfaction with a Discovery Layer at Ryerson University

Abstract

Web-scale discovery systems are becoming prevalent in research libraries. Although a number of studies have explored various impacts of discovery systems, few studies exist on user satisfaction. The investigators of this study evaluated user satisfaction with the discovery service Summon at Ryerson University, using online questionnaires and in-person focus groups. Results indicated a high level of satisfaction overall, although this was heavily influenced by the quality of search results over ease of use. The study provides insight into the information-seeking behavior and search preferences of a user when a discovery layer is implemented in a research library.ith the recent growth of web-scale discovery (WSD) services in academic libraries, both users and library staff alike are adjusting their informationseeking behaviors in response to these new tools. Athena Hoeppner defined WSD as "a preharvested central index coupled with a richly featured discovery layer providing a single search across a library's local, open access, and subscription collections."1 In light of the Google phenomenon, many users have come to expect a 'one-stop' search experience, which is changing how they access library resources and, consequently, the services offered in academic libraries. The expansion of WSD services in academic libraries may represent a move away from simply searching for materials, toward an expectation of accessing materials in full text.2 WSD tools aim to meet user expectations with a single search and access point.In light of these shifting priorities, the evaluation and assessment of WSD services will become increasingly important, as libraries need to determine user satisfaction as a measure of the value of their investment. The following study seeks to determine user satisfaction with a particular WSD tool (Serial Solutions Summon) in a research library at a mid-sized urban university. However, the diverse survey demographic and data gathered make this study relevant for many academic libraries.To contextualize the study, Ryerson University's Library serves a population of over 28,000 students, including about 2,300 graduate students, as well as 780 tenured and tenure-track faculty and approximately 1,700 administrative and support staff. Situated in the heart of downtown Toronto, the library is the only one serving the campus. A research team of three librarians at Ryerson University Library and Archives planned an assessment project for September 2011 to coincide with the release of Summon to the campus for the new academic year. 4 Recognizing the various potential study foci-such as usage, information literacy, usability, and the like-the investigators chose to evaluate user satisfaction to fill a gap in the research literature.With questionnaires being one of the preferred methods of gathering user feedback about electronic resources in libraries, the investigators chose to conduct two online surveys to gather quantitative and qualitative data about user experience with Summon. 5 The online questionnaire...

Framework for Measuring Relevancy in Discovery Environments

“…In one instance, using online questionnaires and in-person focus groups, researchers found users had a high level of satisfaction with their institution's discovery environment, largely attributed to the quality of search results over ease of use. 16 Hamlett and Georgas (2019) conducted a mixed-methods user experience study to understand student perceptions of relevancy in Primo. This study found that participants believed Primo to return relevant results (with an average score of 8.3 out of 10).…”

Web-Scale Discovery: Impact on Library Database Web Page Views and Usage

“…At Illinois State University, they found that most students were able to easily locate needed materials using EBSCO's Web-scale discovery layer, although the authors also cautioned that users will continue to need to access some databases directly as discovery layers cannot fill all the gaps and functionality needed by academic faculty (Williams and Foster 2011). The need to access specific databases for more advanced searching as compared to the broad services of a Web-scale discovery layer was supported in a study at Ryerson University in Toronto where it was found that students, both graduate and undergraduate, are most likely to use the resources their instructors recommend (Lundrigan, Manuel, and Yan 2015). This issue reinforces the need for libraries to market their resources and work with faculty to demonstrate how the library's discovery layer can meet their needs if the library wants to see increased usage.…”

“…While ideally Primo should make searching for library resources more efficient, user friendly, and accessible for a broader range of users, caution should be used, especially when considering more advanced users (Jarrett 2012). In summary, Primo and other Web-scale discovery layers may best match the needs of undergraduates and basic researchers, but do not meet the needs of graduate students and faculty (Lundrigan, Manuel, and Yan 2015;Nichols et al 2014).…”

“…This searching behavior would not be surprising considering Primo delivers a wide variety of results from a myriad of library holdings. The simplicity of searching all these holdings also makes Primo a useful starting point for entry-level searchers who may be unsure of where to begin a search (Lundrigan, Manuel, and Yan 2015). As shown in Figure 3, web page views of databases by subject decreased significantly in 2015, the year after CWU migrated to Primo.…”

Preference vs. Authority: A Comparison of Student Searching in a Subject-Specific Indexing and Abstracting Database and a Customized Discovery Layer

“…11 Lundrigan, Manuel, and Yan found that the majority of respondents to their user satisfaction survey were "very" or "moderately" satisfied with the Summon discovery layer, though this was less true for graduate students and students in certain disciplines. 12 Student participants in Asher, Duke, and Wilson's study indicated a preference for discovery tools and Google Scholar over traditional library tools such as the catalog and databases. 13 As Gross and Sheridan conclude, "there is no doubt that the students found [the webscale discovery tool] an easy way to get results and probably easier than the various options they were previously faced with."…”