An Evaluation Model for the National Consortium of Institutional Repositories of Korean Universities

Abstract

This study, funded by KERIS (The Korean Education and Research InformationService) aims at building an evaluation model for diagnosing and solving problems of the "dCollection" consortium, composed of institutional repositories at more than 40 universities. The evaluation model, as tools not only for a member library's self-evaluation of its own system but also for assessment of the dCollection systems as a whole, is set up in two steps: First, based upon literature review from digital library evaluation and institutional repositories, and case studies on university repositories, the conceptual framework was composed of four categories: content; system and network; uses, users and submitters; and management and policy. And, second, the evaluation framework was modified through three methods, such as a Delphi method, an expert forum and a simulation test, leading to the evaluation model with 4 categories and 34 indicators. IntroductionAccess to academic journals has become more difficult as the prices of academic journals have increased rapidly. Furthermore, the advent of Web-based journals has changed the way institutions subscribe to journals, from choosing individual publications to contracting with publishing companies. These changes have led to a decrease in materials at libraries, resulting in two problems: it becomes more difficult to obtain information, and more difficult to circulate it. Under these circumstances the open access movement started. The movement aims to provide researchers with more access to resources, and authors with wider dissemination of their works, resulting in greater research impact. To realize open access as an alternative to the existing scholarly communications, distributing journals through institutional repositories was proposed.Motivated by the open access movement, academic institutions are building institutional repositories (IRs) for a nationwide knowledge distribution infrastructure. KERIS, the governmental agency which provides various support for university information systems, proposed to organize institutional repositories into a consortium, called "dCollection (Digital Collection)," composed of more than 40 universities since 2003, to give users more access to resources, and to give authors more readers. For this consortium, KERIS also built the repository software platform called "dCollection system," in cooperation with member universities, to freely distribute the platform to the member universities. For the federated searches, KERIS's Research Information Service System, called "RISS," harvests metadata from the dCollection systems operated by the member libraries.The dCollection Promotion Project, funded by KERIS, aims at building an IR evaluation model. In this model the procedural assessment is more emphasized than the actual performance, in order to pinpoint procedural weakness of institutional repositories of each member library and to find their customized solutions, where procedural assessment evaluates current operating and financial processes, while perf...

Indicadores para la evaluación de repositorios institucionales de acceso abierto

“…En la bibliografía revisada se han dividido en seis categorías los indicadores que se proponen: marketing (cómo se promociona el repositorio) (Westell, 2006;Cassella, 2010;Primary Research Group, 2012;Vierkant, 2013); indicadores económicos, es decir, cómo se financia tanto desde la universidad como externamente; interoperabilidad y cooperación con otros repositorios, que está relacionado con la facilidad para ser recolectado y la relación con otras instituciones (Westell, 2006;Swan, Houghton, 2012;Cassella, Morando, 2012;Primary Research Group, 2012); políticas, que indica qué grado de implicación tiene la propia institución en el depósito de documentos de investigación en el repositorio (Kim, Kim, 2006;Westell, 2006;Barrueco Cruz et al, 2010;Vierkant, 2013); contenidos, que indican numéricamente si los investigadores o docentes están realmente concienciados porque introducen sus resultados en el portal (Swan, Houghton, 2012); y, por último, servicios de valor añadido, que supone el esfuerzo de cantidad de personas, y cualificación, por una parte, y por otra los servicios que se dan a los investigadores para que les interese el repositorio como herramienta de difusión de sus investigaciones (Thibodeau, 2007;Cassella, 2010;Vierkant, 2013).…”

“…En una segunda etapa de la investigación, se pretende realizar un estudio a través de encuestas y focus group (Kim, Kim, 2006), para determinar el grado de conocimiento de los usuarios finales (investigadores), y su satisfacción con el repositorio, así como la opinión de los gestores de la investigación dentro de una institución. Con estos datos, se podría conocer cómo un repositorio puede gestionarse de una forma efectiva desde el punto de vista interno, y cómo esa gestión puede influir en la satisfacción del usuario final, de tal forma que considere el repositorio institucional como el camino natural para difundir su investigación.…”

The Case for Standardized Reporting and Assessment Requirements for Institutional Repositories

“…Though some repository software applications do generate limited statistics by analyzing Web logs (e.g., EPrints, D-Space) and some repositories are federating with other institutions to make possible more cross-repository searching and statistical reporting (Kim & Kim, 2006), measuring and analyzing repositories is an activity that requires much more information than simple Web use logs and directory lists can provide (Thomas & McDonald 2007a, b). Additional information is required for effective repository evaluation, such as information about organizational and individual productivity and influence, specified relationships between repository content and external information resources, and indicators of the degree to which a repository is serving the needs of designated communities.…”

Modelo de gestão de repositórios institucionais de acesso aberto à informação científica

“…6. Kim (2010) traz as contribuições mais relevantes para este trabalho. Ela sistematiza um modelo de gerenciamento de repositórios institucionais para a construção de um consórcio nacional.…”

“…O trabalho de maior relevância para a construção do modelo de gestão de repositórios institucionais é o estudo de Kim (2010), que trata do mesmo assunto. Em seu trabalho a autora, apresentou categorias e metodologias de análise de repositórios digitais e bibliotecas digitais.…”