Protocol Analysis of a Federated Search Tool: Designing for Users

Abstract

Summary. Librarians at Springfield College conducted usability testing of Endeavor's federated search tool, ENCompass for Resource Access. The purpose of the testing was to make informed decisions prior to customizing the look and function of the software's interface in order to make the product more usable for their patrons. Protocol, or think-aloud, analysis was selected as a testing and analysis method. Subjects from the general college community were recruited and given a list of tasks to perform on ENCompass, and they were asked to speak all of their thoughts out loud as they worked. Upon analyzing the test results, researchers found that subjects' problems fell into certain categories, such as unfamiliarity with terms or navigation from screen to screen. The researchers were able to use their findings to recommend extensive revisions to the interface, which improved usability for this library's patrons.

Usability Test Results for a Discovery Tool in an Academic Library

“…Research has shown that such a choice has long been in conflict with users' expectations. 1 Federated search software was unable to completely fulfill users' expectations because of its limited technology. 2 Now that discovery tools provide a truly integrated search experience, with greatly improved relevance rankings, response times, and increased consistency, libraries can finally begin to meet this area of user expectation.…”

Applying Hierarchical Task Analysis Method to Discovery Layer Evaluation

“…Emily Alling and Rachel Naismith aggregated common findings from a number of studies involving the traditional federated search technology. [9] Besides slow response time, other key causes of frustrating inefficiency were: limited information about search results, information overload due to the lack of filters, and the fact that results were not ranked in order of relevance (see also [2][1]).…”

“…book cover images, ratings and comments), (4) Faceted navigation for search results, (5) Simple keyword search on every page, (6) More precise relevancy (with circulation statistics a contributing factor), (7) Automatic spell check, (8) Recommendations to related materials (common in commercial sites, e.g. Amazon), (9) Allowing users to add data to records (e.g. reviews), (10) RSS feeds to allow users to follow top circulating books or topic related updates in the library catalogue, (11) Links to social networking sites to allow users to share their resources, (12) Stable URL's that can be easily copied, pasted and shared.…”

Student perceptions of federated searching vs single database searching

“…George (2008) also performed usability testing of a MetaLib interface in order to inform future software development decisions. Alling and Naismith (2007) used a think‐aloud protocol analysis method to gather user feedback about the ENCompass Endeavor federated search tool. The wealth of usability studies of federated search tools confirms that librarians have committed to adopting and improving federated searching.…”