Evaluating sliding and sticky target policies by measuring temporal drift in acyclic walks through a web archive

Abstract

When a user views an archived page using the archive's user interface (UI), the user selects a datetime to view from a list. The archived web page, if available, is then displayed. From this display, the web archive UI attempts to simulate the web browsing experience by smoothly transitioning between archived pages. During this process, the target datetime changes with each link followed; drifting away from the datetime originally selected. When browsing sparselyarchived pages, this nearly-silent drift can be many years in just a few clicks. We conducted 200,000 acyclic walks of archived pages, following up to 50 links per walk, comparing the results of two target datetime policies. The Sliding Target policy allows the target datetime to change as it does in archive UIs such as the Internet Archive's Wayback Machine. The Sticky Target policy, represented by the Memento API, keeps the target datetime the same throughout the walk. We found that the Sliding Target policy drift increases with the number of walk steps, number of domains visited, and choice (number of links available). However, the Sticky Target policy controls temporal drift, holding it to less than 30 days on average regardless of walk length or number of domains visited. The Sticky Target policy shows some increase as choice increases, but this may be caused by other factors. We conclude that based on walk length, the Sticky Target policy generally produces at least 30 days less drift than the Sliding Target policy.

Focused Crawl of Web Archives to Build Event Collections

“…If the Memento is deemed relevant, it is added to the event collection, its outlinks are extracted and added to the priority queue. We note that most web archives rewrite outlinks in their Mementos to point back into the same archive rather than to the live web, even when the archive does not hold a Memento for the linked resource or only holds Mementos that are temporally distant from the desired time, which in our case is DTE [2]. We therefore add the original URI (URI-R) of the outlink, which can be obtained using features of the Memento protocol, to the priority queue rather than the rewritten URI-M of the outlink.…”

Not all mementos are created equal: measuring the impact of missing resources

“…8 When using a page-at-a-time archival service, the resulting memento contains embedded resources with the same archival datetime [1]. This section identifies our damage measurement of this page-at-a-time archiver and outlines the differences between Heritrix and WebCite.…”

“…The Internet Archive alone boasts 455 billion Web pages in its archive, 1 which is far larger than can be evaluated through human methods. While Banos et al constructed the CLEAR method to assign a predictive archivability score [6], a similar score for the actual performance of an archival tool does not exist outside of the simple metric of the percent of embedded resources archived.…”

Analyzing web archives through topic and event focused sub-collections

“…As the entire process is done manually, the researcher will typically have to try many hyperlinks that are not available in the archive. The researcher also has to be aware of temporal drift while navigating the archive [1]. Temporal drift occurs because the linking and linked page were usually crawled at different times and therefore each navigation step moves the analyzed point in time.…”