Maintaining Coherent Views over Dynamic Distributed Data

Abstract

Abstract. Data delivered today over the web reflects rapid and unpredictable changes in the world around us. We are increasingly relying on content that provides dynamic, interactive, personalized experiences. To achieve this, the content of most web pages is created dynamically, by executing queries dynamically, using data that changes dynamically from distributed sources identified dynamically. A typical web page presents a "view" of the world constructed from multiple sources. In this paper, we examine the nature of dynamics of distributed data and discuss fresh approaches to maintain the coherency of the views seen by the users. Achieving such coherency while developing scalable low-overhead solutions poses challenges in terms of delivering data with the required fidelity in spite of data dynamics as well as failures in the infrastructure. How these challenges can be met by the judicious design of algorithms for data dissemination, caching, and cooperation forms the crux of our work.Keywords: Views, dynamic data dissemination, web, internet scale algorithms, data coherencyThe web is becoming a universal medium for information publication and usage. Such information is becoming more and more dynamic and usage is varying from simple tracking to online decision making in real time. Applications include auctions, personal portfolio valuations for financial decisions, route planning based on traffic information, etc. For such applications, data from one or more independent data sources may be aggregated to determine if some action is warranted. Given the increasing number of such applications that make use of highly dynamic data, there is significant interest in systems that can efficiently deliver the relevant updates automatically.In this paper we summarize our work related to the topic of executing continuous queries over dynamic data so that correct results are always presented to users. Specifically, we present low-cost, scalable techniques to answer continuous aggregation queries using a Content Distribution Network (CDN) of dynamic data items. In such a network of data aggregators (DAs), each data aggregator serves a set of data items at specific coherencies. Just as various fragments of a dynamic web-page are served by one or more nodes of a CDN, serving a query involves decomposing a client query into sub-queries and executing sub-queries on judiciously chosen data aggregators with their individual sub-query incoherency bounds.Caches and Views. Seen in the Web context, websites have transitioned from a static content model, where content is served from ready-made files, to a dynamic content model, where content is generated on demand. Dynamic content generation allows sites to offer a wider variety of services and content. But, as Internet traffic continues to grow and websites become increasingly complex, performance and scalability are major issues for website owners and users. Web 3.0 techniques provide website visitors with dynamic, interactive, and personalized experiences. However while serving dyn...