Evaluating digital libraries in the health sector. Part 2: measuring impacts and outcomes

Abstract

This is the second part of a two-part paper which explores methods that can be used to evaluate digital libraries in the health sector. Part 1 focuses on approaches to evaluation that have been proposed for mainstream digital information services. This paper investigates evaluative models developed for some innovative digital library projects, and some major national and international electronic health information projects. The value of ethnographic methods to provide qualitative data to explore outcomes, adding to quantitative approaches based on inputs and outputs is discussed. The paper concludes that new 'post-positivist' models of evaluation are needed to cover all the dimensions of the digital library in the health sector, and some ways of doing this are outlined.

Evaluating digital libraries: a systematised review

“… evaluate digital libraries to identify the key user standards (Masoudnia & Ali‐Ebadi, ) analyse aspects and methods used in the evaluation of digital libraries (Azadi‐Ahmadabadi, ; Cullen, , ; Esfandiyari‐moghadam & Bayat, ; Mehrabadi, ; Noroozi, ; Saracevic, ; Zhang, ) analyse the user interface/usability of digital libraries (Adabi & Jalal‐Dizegi, ; Alipoorhafezi & Adabi, ; Frias‐Martinez, Chen & Liu, ; Jeng, ; Joo & Yeon Lee, ; Noroozi & Akbari, ) analyse search capabilities (Evaluate and compare the information displayed on digital libraries of national content consortium, ; Evaluation of search capabilities of digital libraries applications in Iran, ; Nabavi, ). efficiency of digital libraries (Evaluation of Simorgh Digital Library with emphasis on managerial perspective, ; Tramullas, Sánchez‐Casabón & Garrido‐Picazo, ) investigation of use and access capability (Carlo Bertot, Snead, Jaeger & McClure, ; Snead, Bertot, Jaeger & McClure, ) ontological aspects (Tsakonas, Mitrelis, Papachristopoulos & Papatheodorou, ; Tsakonas & Papatheodorou, ) usefulness and usability (Jabeen, Qinjian, Yihan, Jabeen & Imran, ; Quijano‐Solís & Novelo‐Peña, ; Tsakonas & Papatheodorou, ). …”

“… review (all types of review articles) (Ahmad & Abawajy, ; Alipoorhafezi & Adabi, ; Cullen, , ; Esfandiyari‐moghadam & Bayat, ; Fuhr et al., ; Jose, ; Mehrabadi, ; Noroozi, , 2010a; Noroozi & Akbari, ; Saracevic, ; Tsakonas et al., ; Tsakonas & Papatheodorou, ) checklist (Evaluate and compare the information displayed on digital libraries of national content consortium, ; Evaluation of search capabilities of digital libraries applications in Iran, ; Evaluation of Simorgh Digital Library with emphasis on managerial perspective, ; Hoe‐Lian Goh et al., ; Nabavi, ; Noroozi, 2010b; Noroozi & Montazeri, ; Tripathi & Jeevan, ; Zolghadr & Momeni, ) content analysis (Azadi‐Ahmadabadi, ; Gkoumas & Lazarinis, ; Sohrabi et al., ) online questionnaire and interview (Lai et al., ; Madle et al., ; Zhang, ) Online questionnaire (Garibay et al., ; Tsakonas & Papatheodorou, ) library methods (Retrospective studies that include reviews of articles and books, and questions from expert people) (Hassanzadeh & Sohrabzadeh, ) questionnaire and interview (Moreira et al., ; Snead et al., ) researcher developer questionnaire and checklist (mixed tools) (Amini, ; Ghaebi, Baradar & Farnaghi, ) semiexperimental method (Quijano‐Solís & Novelo‐Peña, )See Figure for the frequency of the methods used. …”

In Pursuit of e-Quality

“…This section draws heavily on three excellent recent reviews. [20][21][22] One important approach seeks to address the question, "How are digital libraries valued by their users?" 20 Derived from the marketing domain and involving work relating to e-service quality (e-SQ), this explores the customer-service angle of Internet-based interactions, not only from the perspective of e-commerce 23 but also as applied to digital library services.…”

Evaluation of outreach services for primary care and mental health; assessing the impact

“…The assessment of impact of library/information services on the business of the organizations which they serve, which in the case of a health service will be improved outcomes for patients, better promotion of good health, and cost saving, is a topic which is generally accepted as vital, but difficult to achieve. Many methods and techniques have been adopted in an attempt to evaluate the impact of library/information services in general; 3–13 none of these, however, has been generally accepted as providing a reliable measure of impact. A number of systematic reviews of impact studies in the specific area of value of information provided through health‐care library services as it impacts patient care have also been published 14–18 .…”