The Design and Use of Assessment Frameworks in Digital Curation

Abstract

To understand and improve their current abilities and maturity, organizations use diagnostic instruments such as maturity models and other assessment frameworks. Increasing numbers of these are being developed in digital curation. Their central role in strategic decision making raises the need to evaluate their fitness for this purpose and develop guidelines for their design and evaluation. A comprehensive review of assessment frameworks, however, found little evidence that existing assessment frameworks have been evaluated systematically, and no methods for their evaluation. This article proposes a new methodology for evaluating the design and use of assessment frameworks. It builds on prior research on maturity models and combines analytic and empirical evaluation methods to explain how the design of assessment frameworks influences their application in practice, and how the design process can effectively take this into account. We present the evaluation methodology and its application to two frameworks. The evaluation results lead to guidelines for the design process of assessment frameworks in digital curation. The methodology provides insights to the designers of the evaluated frameworks that they can consider in future revisions; methodical guidance for researchers in the field; and practical insights and words of caution to organizations keen on diagnosing their abilities.

Case Study: Design of an Approach for Assessing a Novel Health Capability Maturity Model

“…We designed a general assessment approach building on the principles in evaluating capability maturity models developed by Becker et al [10]. We expanded upon these core principles in the context of a sector-wide health care environment that comprised 86 health services.…”

Design features of digital transformation maturity models: a systematic literature analysis and future research directions

“…Therefore, maturity models are tackled in solving challenging issues and developed to assist the companies in diagnosing their current positions (as-is) and finally guiding them to reach advanced levels (to-be or desired) (Becker et al , 2009; De Bruin et al , 2005; Barata et al , 2018) to create competitive power to cut costs and improve quality (De Bruin et al , 2005) and also are effective in the strategic decision-making process, such as investment plans (Becker et al , 2020). Maturity models consist of a set of dimensions and incremental maturity levels.…”