Digital Libraries: A Generic Classification and Evaluation Scheme

Fuhr, Norbert; Hansen, Preben; Mabe, Michael; Micsik, András; Sølvberg, Ingeborg

doi:10.1007/3-540-44796-2_17

Cited by 71 publications

(54 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, his analysis shows that there is no agreement regarding the exact elements of these issues for digital library evaluation. Trying to fill some gaps in this area, Fuhr et al developed a new description scheme using four major dimensions: collection, technology, users and uses [7]. Based on this dimensions, a questionnaire was developed and the need for an appropriate test collection was stated, similar to the TREC and CLEF initiatives.…”

Section: Evaluating Quality In Digital Librariesmentioning

confidence: 99%

Using Semantic Technologies in Digital Libraries – A Roadmap to Quality Evaluation

Tönnies

Balke

2009

Research and Advanced Technology for Digital Libraries

View full text Add to dashboard Cite

Abstract. In digital libraries semantic techniques are often deployed to reduce the expensive manual overhead for indexing documents, maintaining metadata, or caching for future search. However, using such techniques may cause a decrease in a collection's quality due to their statistical nature. Since data quality is a major concern in digital libraries, it is important to be able to measure the (loss of) quality of metadata automatically generated by semantic techniques. In this paper we present a user study based on a typical semantic technique used for automatic metadata creation, namely taxonomies of author keywords and tag clouds. We observed experts assessing typical relations between keywords and documents over a small corpus in the field of chemistry. Based on the evaluation of this experiment, we focused on communalities between the experts' perception and thus draw a first roadmap on how to evaluate semantic techniques by proposing some preliminary metrics.

show abstract

Section: Evaluating Quality In Digital Librariesmentioning

confidence: 99%

Using Semantic Technologies in Digital Libraries – A Roadmap to Quality Evaluation

Tönnies

Balke

2009

Research and Advanced Technology for Digital Libraries

View full text Add to dashboard Cite

show abstract

“…In the 'Content' category, most scholars agree on including size, diversity, quality, selection, connections, and metadata as key criteria (Saracevic, 2000(Saracevic, & 2005Saracevic & Covi, 2000;Fuhr et al, 2001;Kanto, 2005). Saracevic emphasizes more on procedural characteristics of content management, while Fuhr and Kanto do more on formal features of the content.…”

Section: Literature Reviewmentioning

confidence: 99%

“…In the 'System' category, criteria such as access, integration, and reliability are included, among which the most frequently mentioned indicator is access (Saracevic, 2000(Saracevic, & 2005Itsumura, 2000;Fuhr et al, 2001). Functionality (ability to group, limiting), interoperability, and cooperation with other systems are also included as key indicators (Larsen, 2002;Crow 2002b;Kanto, 2005).…”

Section: Literature Reviewmentioning

confidence: 99%

An Evaluation Model for the National Consortium of Institutional Repositories of Korean Universities

Kim

2006

Proc of Assoc for Info

View full text Add to dashboard Cite

This study, funded by KERIS (The Korean Education and Research InformationService) aims at building an evaluation model for diagnosing and solving problems of the "dCollection" consortium, composed of institutional repositories at more than 40 universities. The evaluation model, as tools not only for a member library's self-evaluation of its own system but also for assessment of the dCollection systems as a whole, is set up in two steps: First, based upon literature review from digital library evaluation and institutional repositories, and case studies on university repositories, the conceptual framework was composed of four categories: content; system and network; uses, users and submitters; and management and policy. And, second, the evaluation framework was modified through three methods, such as a Delphi method, an expert forum and a simulation test, leading to the evaluation model with 4 categories and 34 indicators. IntroductionAccess to academic journals has become more difficult as the prices of academic journals have increased rapidly. Furthermore, the advent of Web-based journals has changed the way institutions subscribe to journals, from choosing individual publications to contracting with publishing companies. These changes have led to a decrease in materials at libraries, resulting in two problems: it becomes more difficult to obtain information, and more difficult to circulate it. Under these circumstances the open access movement started. The movement aims to provide researchers with more access to resources, and authors with wider dissemination of their works, resulting in greater research impact. To realize open access as an alternative to the existing scholarly communications, distributing journals through institutional repositories was proposed.Motivated by the open access movement, academic institutions are building institutional repositories (IRs) for a nationwide knowledge distribution infrastructure. KERIS, the governmental agency which provides various support for university information systems, proposed to organize institutional repositories into a consortium, called "dCollection (Digital Collection)," composed of more than 40 universities since 2003, to give users more access to resources, and to give authors more readers. For this consortium, KERIS also built the repository software platform called "dCollection system," in cooperation with member universities, to freely distribute the platform to the member universities. For the federated searches, KERIS's Research Information Service System, called "RISS," harvests metadata from the dCollection systems operated by the member libraries.The dCollection Promotion Project, funded by KERIS, aims at building an IR evaluation model. In this model the procedural assessment is more emphasized than the actual performance, in order to pinpoint procedural weakness of institutional repositories of each member library and to find their customized solutions, where procedural assessment evaluates current operating and financial processes, while perf...

show abstract

“…There are different evaluation approaches like economical, sociological, usercentric, and system-centric [4,14]. User-centric approaches can be used to get feedback by the customers about the relevancy of the results.…”

Section: Introductionmentioning

confidence: 99%

Towards Evaluating the Impact of Ontologies on the Quality of a Digital Library Alerting System

Huhn

Höfner

Kießling

2005

Research and Advanced Technology for Digital Libraries

View full text Add to dashboard Cite

Abstract. Advanced personalization techniques are required to cope with novel challenges posed by attribute-rich digital libraries. At the heart of our deeply personalized alerting system is one extensible preference model that serves all purposes. In this paper we focus on ontology and quality assessment in conjunction with our search technology Preference XPath and XML-based semantic annotations of digital library multimedia objects. We evaluate the impacts of automatic query expansion by ontologies by embedding our alerting system PNews as a black box or a glass box in a test lab. It changes configuration parameters on its own, feeds test cases to P-News, compares the results of different configurations, and stores the result set for further evaluations. The most important indications of this work in progress are: The use of ontologies improves the quality of the result set, generates further results of higher quality, and implies the use of knowledge to reduce a loss of focus.

show abstract

Digital Libraries: A Generic Classification and Evaluation Scheme

Cited by 71 publications

References 3 publications

Using Semantic Technologies in Digital Libraries – A Roadmap to Quality Evaluation

Using Semantic Technologies in Digital Libraries – A Roadmap to Quality Evaluation

An Evaluation Model for the National Consortium of Institutional Repositories of Korean Universities

Towards Evaluating the Impact of Ontologies on the Quality of a Digital Library Alerting System

Contact Info

Product

Resources

About