Caching personalised and database-related dynamic web pages

Chang, Yeim-Kuan; Ting, I-Wei; Lin, Yu‐Liang

doi:10.1504/ijhpcn.2010.037799

Cited by 3 publications

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two approaches used by existing search engines to explore some db-pages. First, search engines may collect cached dbpages, which are generated for some query strings, from the caches of web proxies and web servers [8], [17]. Second, search engines may submit as many trial query strings as possible to web applications to generate db-pages [19].…”

Section: Example 1 (Query String and Db-page)mentioning

confidence: 99%

Dash: A Novel Search Engine for Database-Generated Dynamic Web Pages

Lee

Bankar

Zheng

et al. 2012

2012 IEEE 32nd International Conference on Distributed Computing Systems

View full text Add to dashboard Cite

Abstract-Database-generated dynamic web pages (db-pages, in short), whose contents are created on the fly by web applications and databases, are now prominent in the web. However, many of them cannot be searched by existing search engines. Accordingly, we develop a novel search engine named Dash, which stands for Db-pAge SearcH, to support db-page search. Dash determines db-pages possibly generated by a target web application and its database through exploring the application code and the related database content and supports keyword search on those db-pages. In this paper, we present its system design and focus on the efficiency issue.To minimize costs incurred for collecting, maintaining, indexing and searching a massive number of db-pages that possibly have overlapped contents, Dash derives and indexes db-page fragments in place of db-pages. Each db-page fragment carries a disjointed part of a db-page. To efficiently compute and index db-page fragments from huge datasets, Dash is equipped with MapReduce based algorithms for database crawling and db-page fragment indexing. Besides, Dash has a top-k search algorithm that can efficiently assemble db-page fragments into db-pages relevant to search keywords and returns the k most relevant ones. The performance of Dash is evaluated via extensive experimentation.

show abstract

Section: Example 1 (Query String and Db-page)mentioning

confidence: 99%

Dash: A Novel Search Engine for Database-Generated Dynamic Web Pages

Lee

Bankar

Zheng

et al. 2012

2012 IEEE 32nd International Conference on Distributed Computing Systems

View full text Add to dashboard Cite

show abstract

Surviving a search engine overload

Koehl

Wang

2012

Proceedings of the 21st International Conference on World Wide Web

View full text Add to dashboard Cite

Search engines are an essential component of the web, but their web crawling agents can impose a significant burden on heavily loaded web servers. Unfortunately, blocking or deferring web crawler requests is not a viable solution due to economic consequences. We conduct a quantitative measurement study on the impact and cost of web crawling agents, seeking optimization points for this class of request. Based on our measurements, we present a practical caching approach for mitigating search engine overload, and implement the two-level cache scheme on a very busy web server. Our experimental results show that the proposed caching framework can effectively reduce the impact of search engine overload on service quality.

show abstract