Web archives as a data resource for digital scholars

Abstract

The aim of this article is to provide an exploratory analysis of the landscape of web archiving activities in Europe. Our contribution, based on desk research, and complemented with data from interviews with representatives of European heritage institutions, provides a descriptive overview of the state-of-the-art of national web archiving in Europe. It is written for a broad interdisciplinary audience, including cultural heritage professionals, IT specialists and managers, and humanities and social science researchers. The legal, technical and operational aspects of web archiving and the value of web archives as born-digital primary research resources are both explored. In addition to investigating the organisations involved and the scope of their web archiving programmes, the curatorial aspects of the web archiving process, such as selection of web content, the tools used and the provision of access and discovery services are also considered. Furthermore, general policies related to web archiving programmes are analysed. The article concludes by offering four important issues that digital scholars should consider when using web archives as a historical data source. Whilst recognising that this study was limited to a sample of only nine web archives, this article can nevertheless offer some useful insights into the technical, legal, curatorial and policy-related aspects of web archiving. Finally, this paper could function as a stepping stone for more extensive and qualitative research.

‘Go fish’: Conceptualising the challenges of engaging national web archives for digital research

“…These challenges have been collated elsewhere, including Maemura (2018) who synthesises common problems, such as defining a corpus of study within web archival collections, evaluating the suitability of collected materials and considering the implications of ethics and consent. Most recently, Vlassenroot et al (2019) review the landscape of web archiving activities within the European context, and summarise four 'important considerations' for digital scholarship, including: how and why selection activities take place, the access conditions and legal frameworks that govern web archives, and the 'high level of technical knowledge' required to contextualise and make use of collections data. Whilst Vlassenroot et al connect some of the challenges of researcher use with the very nature of the archival process, little work has empirically and comparatively addressed how researcher engagement is intricately connected to the complex processes of web archival scoping, collection and curation, in practice.…”

Web-archiving and social media: an exploratory analysis

“…Vlassenroot et al (Vlassenroot et al, 2019 , Table 2 ) reported that a number of web archiving initiatives in their study included social media content in their collections; however, the policies with regard to social media differed widely between institutions. Table 2 provides an update of this overview, including data from additional SMA initiatives (updated data is marked in bold).…”

“…While most archiving institutions use a twofold approach for archiving regular web content – combining broad crawls (covering top-level domains) and selective crawls (for thematic or events-based collections) (Vlassenroot et al, 2019 ), the nature of social media content (e.g. the volume, velocity and variety in which content is produced) necessitates another, more targeted approach.…”

“…There is a strong need for provenance metadata on different levels for archived web content (Venlet et al, 2018 ) for both basic users and scholars (Littman et al, 2018 ; Vlassenroot et al, 2019 ); this is often in contrast to the needs of practitioners (Venlet et al, 2018 ). In case of social media this metadata can be provided via Application Programming Interfaces (APIs).…”

“…The aim of the article is threefold. First we aim to extend the most recent descriptive state-of-the-art article on national web archiving (Vlassenroot et al, 2019 ) by complementing it with a review of SMA practices. Secondly, we outline the legal, technical and operational (such as the selection and preservation policy) aspects of archiving social media content.…”