Organizational assessment frameworks for digital preservation: A literature review and mapping

Abstract

As the field of digital preservation matures, there is an increasing need to systematically assess an organization's abilities to achieve its digital preservation goals, and a wide variety of assessment tools have been created for this purpose. To map the landscape of research in this area, evaluate the current maturity of knowledge on this central question in DP and provide direction for future research, this paper reviews assessment frameworks in digital preservation through a systematic literature search and categorizes the literature by type of research. The analysis shows that publication output around assessment in digital preservation has increased markedly over time, but most existing work focuses on developing new models rather than rigorous evaluation and validation of existing frameworks. Significant gaps are present in the application of robust conceptual foundations and design methods, and in the level of empirical evidence available to enable the evaluation and validation of assessment models. The analysis and comparison with other fields suggest that the design of assessment models in DP should be studied rigorously in both theory and practice, and that the development of future models will benefit from applying existing methods, processes, and principles for model design.

The Design and Use of Assessment Frameworks in Digital Curation

“…This raises the need to evaluate their fitness for this purpose and to use insights from their application to support their systematic design (Wendler, ). However, most frameworks in digital curation do not disclose evidence about their application, evaluation, and validation (Maemura, Moles, & Becker, ). While the literature on maturity models more generally provides principles and requirements for the design and evaluation of assessment frameworks, it does not provide a comprehensive methodology.…”

“…Correspondingly, the last decade has seen a steady increase in the development of frameworks that are used in practice to assess an organization's digital curation and preservation abilities (Maemura et al, ). Certification frameworks such as the Repositories Audit & Certification checklist (ISO, ) focus on compliance with strict criteria.…”

“…In practice, various combinations of “maturity” and “capability” are used to denote such models and the approaches used to design and apply them, although these concepts have slightly different semantics (Maier, ). We use the terms model or maturity model to refer to the dimensions and criteria that focus and guide the assessment; and the terms framework and assessment framework to refer to the entire set of components within one approach, including the maturity model, an assessment tool or method, and supporting documentation and guidelines (Maemura et al, ).…”

“…Despite the importance of assessment for strategic planning in digital curation, however, a comprehensive review showed that very few frameworks have been empirically evaluated and documented. As a result, little is known about how different frameworks fare in supporting an organization's goals or facilitating improvement (Maemura et al, ). Some frameworks provide narrative reports of their application, but none provides comprehensive evidence about its characteristics and the methods used to evaluate it.…”

“…Case studies published by the creators of frameworks often focus on illustrating and advocating their use (Wendler, ). As a consequence, the design of maturity models in digital curation lacks robust methods that allow us to “scrutinize … these approaches critically for the benefit of the professional community” (Maemura et al, , p. 1632). This article contributes to closing this gap by establishing an evaluation method built on rigorous approaches from other disciplines.…”