Maintenance of data cubes and summary tables in a warehouse

Abstract

Data warehouses contain large amounts of information, often collected from a variety of independent sources. Decisionsupport functions in a warehouse, such as on-line analytical processing (OLAP), involve hundreds of complex aggregate queries over large volumes of data. It is not feasible to compute these queries by scanning the data sets each time, Warehouse applications therefore build a large number of urnrnary tables, or materialized aggregate views, to help them increase the system performance..4s changes, most notably new transactional data, are collected at the data sources, all summary tables at the warehouse that depend upon this data need to be updated. Usually, source changes are loaded into the warehouse at regular intervals, usually once a day, in a batch window, and the warehouse is made unavailable for querying while it is updated. Since the number of summary tables that need to be maintained is often large, a critical issue for data warehousing is how to maintain the summary tables efficiently.in this paper we propose a method of maintaining aggregate views (the surnmary-cieita table method), and use it to solve two problems in maintaining summary tables in a warehouse: ( 1) how to efficiently maintain a summary table while minimizing the batch window needed for maintenance, and (2) how to maintain a large set of summary tables defined over the same base tables.Vvhile several papers have addressed the issues relating to choosing and materializing a set of summary tables, this is the first paper to address maintaining summary tables efficiently. introductionData warehouses contain information that is collected from muh iple, independent data sources and integrated into a common repository for querying and analysis. Often, data warehouses are designed for on-line anaiytica[ processing (OLAP ), where the queries aggregate large volumes of data " \Workperformed wh]le at New Jersey Institute of Technology.Permissionto make digitsl/hard copy of psrt or all this work for personal or claasroomuse ia grantad without fee provided that copies sre not made or diatributadfor profit or commercialadvsntage, the copyright notice, the title of the publicationand its date appear, and notice is given that copying ia by permiaaion of ACM, Inc. To copy otherwise, to republish, to post on sarvers, or to redistributeto Iiats,requires prior specific permission and/or a fee. SIGMOD '97 AZ, USA @ 1997 ACM 0-89791-91

Monotonic complements for independent data warehouses

“…Furthermore, as mentioned in the Introduction, OLAP is a major application domain for data warehousing, where analysts execute complex queries involving aggregate views defined on fact tables. Although aggregate queries cannot be exploited when computing complements, they do not restrict the applicability of our approach either: the fact tables can be maintained as described above using PSJ views, whereas view maintenance algorithms for aggregate queries, e.g., [10,15,24], can be used to maintain materialized aggregate queries.…”

Integrated model-driven dashboard development

“…In other words, an MDBT operation model is a truly computation independent model. Change management has been studied in various different contexts (Mumick et al 1997;Palpanas et al 2002;Marian et al 2001). In all the cases, the main goal is to reduce the time and effort needed for incorporating the changes to a minimum.…”

Mapping Adaptation under Evolving Schemas

“…None of this work investigated how views were affected when the schema is modified. Incremental view maintenance [5,19] is a problem closely related to ours and deals with the methods for efficiently updating materialized views when the base schema data are updated. View adaptation [9,18] is a variant of view maintenance that investigates methods of keeping the data in a materialized view up-to-date in response to changes in the view definition itself.…”