The OAI-PMH static repository and static repository gateway

Hochstenbach, Patrick; Jerez, Henry; Sompel, Herbert Van de

doi:10.1109/jcdl.2003.1204865

Cited by 8 publications

(7 citation statements)

References 7 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The schemas and examples are detailed in [9]. The Opal prototype exposes these XML structures through Static Repositories [10], which are registered at the Los Alamos National Laboratory Experimental OAI Static Repository Gateway (http://libtest.lanl.gov/). Static repositories allow for rapid prototyping, but their file size limitations of 2MB would prevent them from being used in large scale Opal implementations.…”

Section: Figure 6: Opal Harvesting Hierarchymentioning

confidence: 99%

Just-in-time recovery of missing web pages

Harrison

Nelson

2006

Proceedings of the Seventeenth Conference on Hypertext and Hypermedia

View full text Add to dashboard Cite

We present Opal, a light-weight framework for interactively locating missing web pages (http status code 404). Opal is an example of "in vivo" preservation: harnessing the collective behavior of web archives, commercial search engines, and research projects for the purpose of preservation. Opal servers learn from their experiences and are able to share their knowledge with other Opal servers by mutual harvesting using the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). Using cached copies that can be found on the web, Opal creates lexical signatures which are then used to search for similar versions of the web page. We present the architecture of the Opal framework, discuss a reference implementation of the framework, and present a quantitative analysis of the framework that indicates that Opal could be effectively deployed.

show abstract

Section: Figure 6: Opal Harvesting Hierarchymentioning

confidence: 99%

Just-in-time recovery of missing web pages

Harrison

Nelson

2006

Proceedings of the Seventeenth Conference on Hypertext and Hypermedia

View full text Add to dashboard Cite

show abstract

“…Very recently a new approach has been presented to further lower the level of complexity of the OAI-PMH protocol (Hochstenbach, Jerez & Van de Sompel, 2003). This approach allows a data provider to make a metadata collection available on a Web server as a XML file with a specific format.…”

Section: Technical Solutions For Supporting Open Archivesmentioning

confidence: 99%

Open archive solutions to traditional archive/library cooperation

Castelli¹

2003

LIBER

View full text Add to dashboard Cite

The diffusion of Internet is changing the role of archives and libraries and it is opening a wide range of new possibilities. The new vision is that in few years it will be possible to cross-access multiple libraries, archives, museums, and data repositories. The implementation of this revolutionary vision requires the solution of a number of technical, organizational, sociological, and economical issues. Recently, the proposal of a new, low cost technical solution for open repositories of pre-print material, the Open Archives Initiative Protocol for Metadata Harvesting ( OAI-PMH), has greatly stimulated the discussion about these issues in many application frameworks. This paper overviews the status of this discussion in the library and conventional archives frameworks.

show abstract

“…OAI-PMH [23,42] is the de facto standard for metadata interchange within the digital library community, in part because of its rich semantics. Packages for implementing OAI-PMH repositories for XML files have been described [19,37], but they are focused on highly constrained scenarios, not general web content and they do not integrate directly into the web server. document.…”

Section: Introductionmentioning

confidence: 99%

Efficient, automatic web resource harvesting

Nelson

Smith

Campo

2006

Proceedings of the 8th Annual ACM International Workshop on Web Information and Data Management

View full text Add to dashboard Cite

There are two problems associated with conventional web crawling techniques: a crawler cannot know if all resources at a non-trivial web site have been discovered and crawled ("the counting problem") and the human-readable format of the resources are not always suitable for machine processing ("the representation problem"). We introduce an approach that solves these two problems by implementing support for both the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) and MPEG-21 Digital Item Declaration Language (DIDL) into the web server itself. We present the Apache module "mod oai", which can be used to address the counting problem by listing all valid URIs at a web server and efficiently discovering updates and additions on subsequent crawls. Our experiments indicated comparable performance for initial crawls, and dramatic increases in update speed. mod oai can also be used to address the representation problem by providing "preservation ready" versions of web resources aggregated with their respective forensic metadata in MPEG-21 DIDL format.

show abstract

The OAI-PMH static repository and static repository gateway

Cited by 8 publications

References 7 publications

Just-in-time recovery of missing web pages

Just-in-time recovery of missing web pages

Open archive solutions to traditional archive/library cooperation

Efficient, automatic web resource harvesting

Contact Info

Product

Resources

About