A Framework for Aggregating Private and Public Web Archives

Abstract

Personal and private Web archives are proliferating due to the increase in the tools to create them and the realization that Internet Archive and other public Web archives are unable to capture personalized (e.g., Facebook) and private (e.g., banking) Web pages. We introduce 1 a framework to mitigate issues of aggregation in private, personal, and public Web archives without compromising potential sensitive information contained in private captures. We amend Memento syntax and semantics to allow TimeMap enrichment to account for additional attributes to be expressed inclusive of the requirements for dereferencing private Web archive captures. We provide a method to involve the user further in the negotiation of archival captures in dimensions beyond time. We introduce a model for archival querying precedence and short-circuiting, as needed when aggregating private and personal Web archive captures with those from public Web archives through Memento. Negotiation of this sort is novel to Web archiving and allows for the more seamless aggregation of various types of Web archives to convey a more accurate picture of the past Web.

Aggregator Reuse and Extension for Richer Web Archive Interaction

“…The process of HTTP requests as recursively applied through an aggregator subsequently querying additional sources resembles a graph structure, typically reduced to a tree in the conventional case (Section 4.2). As this work reiterates the potential for an aggregator querying an aggregator [16], the scenario arises of graph-style cycles (Figure 3) that must be mitigated. Additionally, we may encounter redundancies in this "chaining" process (Figure 5) where aggregators down the request chain are configured to query identical, previously queried archives with the same parameters.…”

“…For MemGator, however, the set of endpoints is user-configurable, and thus this valid scenario may arise and has implications. The merits of "aggregator chaining" were discussed in the seminal work introducing the concept [16], but did not go into detail or highlight some problems that may occur. We reiterate and address these in Section 6.…”

“…These exhibitions may be strategically and efficiently enhanced through consideration of this potential additional capability for the sake of enhancing the role of the aggregator in use cases for web archives. We build on existing work in defining a framework for aggregating public and private web archives [16]. Our focus will be on identifying (Section 6) and mitigating (Section 7) some outstanding issues both introduced by the framework as well as those that exist in current practice of interfacing with web archives using Memento aggregation.…”

The Past Web

“…If users want to run their own aggregator, they can install MemGator , developed by Old Dominion University. Research by Kelly et al extends aggregators to query both public and private web archives (Kelly et al, 2018), allowing users to seamlessly transition between archives at different levels of access. Without the standardised interfaces offered by the Memento Protocol, such aggregators would need to apply an assortment of hacks specific to each web archive, like those mentioned in Section 1.1.…”

Profiling Web Archival Voids for Memento Routing

“…Hallak estimated recently that almost two thirds of the web traffic is not publicly archivable because it goes to sites that are behind session walls or paywalls, to which some social media sites are big contributors of [22]. Kelly et al developed a framework to archive the private web and integrate it with the public web to fill some of these cavities [23], [24]. These works identify Archival Voids as they show some biases in web archiving as well as quantify the small portion of the web many archives hold.…”