Using neighbors to date web documents

Nunes, Sérgio; Ribeiro, Cristina; David, Gabriel

doi:10.1145/1316902.1316924

Cited by 19 publications

(12 citation statements)

References 21 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies estimate that from 35% to 64% of web documents have valid last-modified dates [14], but these percentages can be significantly improved by using the dates of the web document's neighbors, especially of web resources embedded in the selected document (e.g. images, CSS, JavaScript) [25]. Nevertheless, for simplification, in this work we adopted the crawling date.…”

Section: Temporal Intervalsmentioning

confidence: 99%

Learning temporal-dependent ranking models

Costa¹,

Couto

Silva

2014

Proceedings of the 37th International ACM SIGIR Conference on Research &Amp; Development in Information Retrieval

View full text Add to dashboard Cite

Web archives already hold together more than 534 billion files and this number continues to grow as new initiatives arise. Searching on all versions of these files acquired throughout time is challenging, since users expect as fast and precise answers from web archives as the ones provided by current web search engines. This work studies, for the first time, how to improve the search effectiveness of web archives, including the creation of novel temporal features that explore the correlation found between web document persistence and relevance. The persistence was analyzed over 14 years of web snapshots. Additionally, we propose a temporal-dependent ranking framework that exploits the variance of web characteristics over time influencing ranking models. Based on the assumption that closer periods are more likely to hold similar web characteristics, our framework learns multiple models simultaneously, each tuned for a specific period. Experimental results show significant improvements over the search effectiveness of single-models that learn from all data independently of its time. Thus, our approach represents an important step forward on the state-of-the-art IR technology usually employed in web archives.

show abstract

Section: Temporal Intervalsmentioning

confidence: 99%

Learning temporal-dependent ranking models

Costa¹,

Couto

Silva

2014

Proceedings of the 37th International ACM SIGIR Conference on Research &Amp; Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…We will discuss in details the temporal language model in the next section. Non-learning methods are presented in [78,82,94]. They require an explicit timetagged document.…”

Section: Related Workmentioning

confidence: 99%

“…In the end, the event-time period of the document is generated by assembling all nearly dates to the reference date where their relevancy must be greater than a threshold. Nunes et al [94] propose an alternative approach to dating a non-timestamped document using its neighbors, such as 1) documents containing links to the non-timestamped document (incoming links), 2) documents pointed to the non-timestamped document (outgoing links) and 3) the media assets (e.g., images) associated with the non-timestamped document. They compute the average of last-modified dates extracted from neighbor documents and use it as the time for the non-timestamped document.…”

Section: Related Workmentioning

confidence: 99%

Time-aware approaches to information retrieval

Kanhabua

2012

SIGIR Forum

View full text Add to dashboard Cite

In this thesis, we address major challenges in searching temporal document collections. In such collections, documents are created and/or edited over time. Examples of temporal document collections are web archives, news archives, blogs, personal emails and enterprise documents. Unfortunately, traditional IR approaches based on term-matching only can give unsatisfactory results when searching temporal document collections. The reason for this is twofold: the contents of documents are strongly time-dependent, i.e., documents are about events happened at particular time periods, and a query representing an information need can be time-dependent as well, i.e., a temporal query.Our contributions in this thesis are different time-aware approaches within three topics in IR: content analysis, query analysis, and retrieval and ranking models. In particular, we aim at improving the retrieval effectiveness by 1) analyzing the contents of temporal document collections, 2) performing an analysis of temporal queries, and 3) explicitly modeling the time dimension into retrieval and ranking.Leveraging the time dimension in ranking can improve the retrieval effectiveness if information about the creation or publication time of documents is available. In this thesis, we analyze the contents of documents in order to determine the time of non-timestamped documents using temporal language models. We subsequently employ the temporal language models for determining the time of implicit temporal queries, and the determined time is used for re-ranking search results in order to improve the retrieval effectiveness.We study the effect of terminology changes over time and propose an approach to handling terminology changes using time-based synonyms. In addition, we propose different methods for predicting the effectiveness of temporal queries, so that a particular query enhancement technique can be performed to improve the overall performance. When the time dimension is incorporated into ranking, documents will be ranked according to both textual and temporal similarity. In this case, time uncertainty should also be taken into account. Thus, we propose a ranking model that considers the time uncertainty, and improve ranking by combining multiple features using learning-to-rank techniques.Through extensive evaluation, we show that our proposed time-aware approaches outperform traditional retrieval methods and improve the retrieval effectiveness in searching temporal document collections.

show abstract

“…[5] proposed a novel technique for the estimation of last modification date of a Web page, which complements the traditional approaches based on HTTP headers, by also looking at a document's neighborhood. The neighborhood of a document, as mentioned in the paper, is its incoming links, outgoing links and media files, e.g., pdf files, image files, sound files etc.…”

Section: Neighborhood Techniquementioning

confidence: 99%

Aiding web crawlers; projecting web page last modification

Anjum

2012

2012 15th International Multitopic Conference (INMIC)

View full text Add to dashboard Cite

Due to colossal amount of data on the Web, Web archivists typically make use of Web crawlers for automated collection. The Internet Archive is the largest organization based on a crawling approach in order to maintain an archive of the entire Web. The most important requirement of a Web crawler, specially when they are used for Web archiving, is to be aware of the date (and time) of last modification of a Web page. This strategy has various advantages, most important of them include i) presentation of an up-to-date version of a Web page to the end user ii) ease of adjusting the crawl rate that allows future retrieval of a Web page's version at a given date, or to compute its refresh rate. The typical way for this modification information of a Web page, that is, to use the Last-Modified: HTTP header, unfortunately does not provide correct information every time. In this work, we discuss various techniques that can be used to determine the date of last modification of a Web page with the help of experiments. This will help in adjusting the crawl rate for a specific page and also helps in presenting users with up to date information and thus allowing future versioning of a Web page more meticulous.

show abstract

Using neighbors to date web documents

Cited by 19 publications

References 21 publications

Learning temporal-dependent ranking models

Learning temporal-dependent ranking models

Time-aware approaches to information retrieval

Aiding web crawlers; projecting web page last modification

Contact Info

Product

Resources

About