Scalable decision support for digital preservation: an assessment

Abstract

Structured Abstract:Purpose -Scalable decision support and business intelligence capabilities are required to effectively secure content over time. This article evaluates a new architecture for scalable decision making and control in preservation environments for its ability to address five key goals: (1) Scalable content profiling, (2) Monitoring of compliance, risks and opportunities, (3) Efficient creation of trustworthy plans, (4) Context awareness, and (5) Loosely-coupled preservation ecosystems.Design/methodology/approach -We conduct a systematic evaluation of the contributions of the SCAPE Planning and Watch suite to provide effective and scalable decision support capabilities. We discuss the quantitative and qualitative evaluation of advancing the state of art and report on a case study with a national library.Findings -The system provides substantial capabilities for semi-automated, scalable decision making and control of preservation functions in repositories. Well-defined interfaces allow a flexible integration with diverse institutional environments. The free and open nature of the tool suite further encourages global take-up in the repository communities.Research limitations/implications -The article discusses a number of bottlenecks and factors limiting the realworld scalability of preservation environments. This includes data-intensive processing of large volumes of information, automated quality assurance for preservation actions, and the element of human decision making. We outline open issues and future work.Practical implications -The open nature of the software suite enables stewardship organizations to integrate the components with their own preservation environments and to contribute to the ongoing improvement of the systems.Originality/value -The paper reports on innovative research and development to provide preservation capabilities. The results of the assessment demonstrate how the system advances the control of digital preservation operations from ad-hoc decision making to proactive, continuous preservation management, through a contextaware planning and monitoring cycle integrated with operational systems. Keywords:Repositories, preservation planning, preservation watch, monitoring, scalability, digital libraries. Scalable Decision Support for Digital Preservation: An AssessmentThis article presents a systematic assessment and evaluation of the SCAPE decision support environment comprising PLATO, SCOUT and c3po. We discuss the improvements and identified limitations of the presented system. We furthermore discuss the quantitative and qualitative evaluation of advancing the state of art and report on a case study with a national library. Finally, we summarize the contributions and provide an outlook on future work.

Improving data quality in large-scale repositories through conflict resolution

“…C3PO uses MongoDB as storage and queries the data using map-reduce jobs. While this can in principle provide highly scalable analytics, previous research on a very large test data set highlighted that the platform was not sufficiently scalable for large data sets [7,29].…”

“…For example, the File Information Toolset (FITS) 1 fuses multiple tool outputs and marks any contradictions between the tools in the combined output it produces. The content profiling platform C3PO exposes these conflicts so that they cannot be missed, but does not resolve them [7].…”

Scalable decision support for digital preservation

“…In Becker et al (2015), we will conduct a systematic assessment of the system based on the design goals outlined in this article. We will discuss the improvements of the presented work and identified limitations, based on a quantitative and qualitative evaluation including a case study with a national library.…”

“…Section 4 presents the suite of automated tools designed and developed to improve decision support and control in real-world settings. Becker et al (2015) will discuss the improvements of the presented work and identified limitations, based on a quantitative and qualitative evaluation of advancing the state of art including a case study with a national library.…”

Format technology lifecycle analysis

“…This is increasingly feasible (Holden, 2012) and has been done for a substantial set of the UK Web Archive (Jackson, 2012). For example, a dataset from the Danish web covers 8 years in 441 million files (Becker, Faria, & Duretec, 2015). It contains in-depth, detail-rich feature sets extracted by the File Information Toolset (FITS), but covers fewer years.…”