Evaluating Web Archive Search Systems

Costa, Miguel; Silva, Mário J.

doi:10.1007/978-3-642-35063-4_32

Cited by 10 publications

(18 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Research in temporal IR aims to exploit temporal information in documents and queries for better query understanding and time-based ranking [1,16,32]. Costa and Silva [21] created a temporal test collection from the Portuguese Web Archive [28], to enable evaluation of temporal methods in IR. A test collection consists of queries (topics), documents, and the judgments by users of their relevance to the queries.…”

Section: Related Workmentioning

confidence: 99%

“…Including the temporal aspect of Web archives into retrieval models was discussed in [21]. In their model, they linearly combined a prior which favors documents with more versions or longer existence (time span between first version and last version) with known IR models.…”

Section: Collapsing Versions (Url-based)mentioning

confidence: 99%

“…Previous studies showed that applying existing Information Retrieval (IR) models on Web archives leads to unsatisfactory results [17,21]. Measuring the effectiveness of IR systems can be done using test collections.…”

Section: Introductionmentioning

confidence: 99%

“…A test collection consists of a set of topics (queries), a document collection, and a set of relevance assessments. Costa and Silva [21] extended this approach by taking the characteristics of Web archives into account. Their approach includes the design of a test collection and constructing topics from the users' query log of a functioning Web archive search system.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Quantifying retrieval bias in Web archive search

et al. 2017

View full text Add to dashboard Cite

A Web archive usually contains multiple versions of documents crawled from the Web at different points in time. One possible way for users to access a Web archive is through full-text search systems. However, previous studies have shown that these systems can induce a bias, known as the retrievability bias, on the accessibility of documents in community-collected collections (such as TREC collections). This bias can be measured by analyzing the distribution of the retrievability scores for each document in a collection, quantifying the likelihood of a document's retrieval. We investigate the suitability of retrievability scores in retrieval systems that consider every version of a document in a Web archive as an independent document. We show that the retrievability of documents can vary for different versions of the same document and that retrieval systems induce biases to different extents. We quantify this bias for a retrieval system which is adapted to handle multiple versions of the same document. The retrieval system indexes each version of a document independently, and we refine the search results using two techniques to aggregate similar versions. The first approach is to collapse similar versions of a document based on content similarity. The second approach is to collapse all versions of the same document based on their URLs. In both cases, we found that the degree of bias Universiteit Utrecht, Utrecht, The Netherlands is related to the aggregation level of versions of the same document. Finally, we study the effect of bias across time using the retrievability measure. Specifically, we investigate whether the number of documents crawled in a particular year correlates with the number of documents in the search results from that year. Assuming queries are not inherently temporal in nature, the analysis is based on the timestamps of documents in the search results returned using the retrieval model for all queries. The results show a relation between the number of documents per year and the number of documents retrieved by the retrieval system from that year. We further investigated the relation between the queries' timestamps and the documents' timestamps. First, we split the queries into different time frames using a 1-year granularity. Then, we issued the queries against the retrieval system. The results show that temporal queries indeed retrieve more documents from the assumed time frame. Thus, the documents from the same time frame were preferred by the retrieval system over documents from other time frames.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Collapsing Versions (Url-based)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Quantifying retrieval bias in Web archive search

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The proposed pre-processement algorithm also reduces the number of links in the web graph. An evaluation of ranking algorithms to support temporal search has been undertaken [7]. This demonstration aims to expose how the chosen algorithms perform in practice on a live service.…”

Section: Historical Web Graph Computationmentioning

confidence: 99%

Search the past with the portuguese web archive

Gomes¹,

Cruz²,

Miranda³

et al. 2013

Proceedings of the 22nd International Conference on World Wide Web

Self Cite

View full text Add to dashboard Cite

The web was invented to quickly exchange data between scientists, but it became a crucial communication tool to connect the world. However, the web is extremely ephemeral. Most of the information published online becomes quickly unavailable and is lost forever. There are several initiatives worldwide that struggle to archive information from the web before it vanishes. However, search mechanisms to access this information are still limited and do not satisfy their users who demand performance similar to live-web search engines.This demo presents the Portuguese Web Archive, which enables search over 1.2 billion files archived from 1996 to 2012. It is the largest full-text searchable web archive publicly available [17]. The software developed to support this service is also publicly available as a free open source project at Google Code, so that it can be reused and enhanced by other web archivists. A short video about the Portuguese Web Archive is available at vimeo.com/59507267. The service can be tried live at archive.pt.

show abstract