Not all mementos are created equal: measuring the impact of missing resources

Abstract

Web archives do not always capture every resource on every page that they attempt to archive. This results in archived pages missing a portion of their embedded resources. These embedded resources have varying historic, utility, and importance values. The proportion of missing embedded resources does not provide an accurate measure of their impact on the Web page; some embedded resources are more important to the utility of a page than others. We propose a method to measure the relative value of embedded resources and assign a damage rating to archived pages as a way to evaluate archival success. In this paper, we show that Web users' perceptions of damage are not accurately estimated by the proportion of missing embedded resources. In fact, the proportion of missing embedded resources is a less accurate estimate of resource damage than a random selection. We propose a damage rating algorithm that provides closer alignment to Web user perception, providing an overall improved agreement with users on memento damage by 17 % and an improvement by 51 % if the mementos have a damage rating delta >0.30. We use our algorithm to measure damage in the Internet Archive, showing that it is getting better at mitigating damage over time (going from a damage ). However, we show that a greater number of important embedded resources (2.05 per memento on average) are missing over time. Alternatively, the damage in WebCite is increasing over time (going from 0.375 in 2007 to 0.475 in 2014), while the missing embedded resources remain constant (13 % of the resources are missing on average). Finally, we investigate the impact of JavaScript on the damage of the archives, showing that a crawler that can archive JavaScript-dependent representations will reduce memento damage by 13.5 %.

Preserving Data Journalism: A Systematic Literature Review

“…This set of challenges was followed by lack of knowledge (i.e., education on preservation), and the complexity of objects. The complexity of objects includes challenges associated with objects that are composed of many layers (Broussard and Boss 2018) may be dependent on external resources such as APIs (Boss and Broussard 2017) or embedded resources (Brunelle et al 2015(Brunelle et al , 2016(Brunelle et al , 2017. Several studies noted that knowledge about which objects to preserve in the first place was lacking (Barwick, Dearnley, and Muir 2011;Broussard 2015a), or understanding how to deal with versions (i.e., which version of an object is definitive?…”

“…This category includes recommendations related to the maintenance of authenticity and integrity of digital objects as they are being kept in long-term storage. It further includes recommendations to test web archives for missing embedded resources (Brunelle et al 2015(Brunelle et al , 2016Kelly, Nelson, and Weigle 2014) and the need for development of validation methods for software preservation (Barateiro et al 2012).…”

Adding the Dimension of Time to HTTP

“…Although the gold standard for assessing web archiving quality is still human interaction with a Memento to ensure all embedded resources, links, and functionality are preserved, this level of assessment is clearly not scalable. We have been involved with a range of automated evaluations of the web archiving process, including the Archival Acid Test (Kelly et al, 2014a), which evaluates the capabilities of crawling and playback technology stacks (e.g., the Heritrix crawler (Mohr et al, 2004) and the Wayback Machine playback engine), and assessing Memento damage (Brunelle et al, 2014(Brunelle et al, , 2015 which provides weights to missing embedded resources based on heuristics for determining if the missing resource was 'important'.…”

What's cached is prologue: Reviewing recent web archives research towards supporting scholarly use

“…I focus here on studies that compare coverage between collections for a particular research topic or domain, as opposed to general quantitative evaluation of an archive's coverage compared to what exists on the live web (e.g. Ainsworth et al, 2011;Ainsworth et al, 2015;Brunelle et al, 2015). For example, Brügger (2013a) considers the coverage of material relating to Danish parliamentary elections by comparing historical network graphs available from the Danish Netarkivet collection and the Internet Archive.…”