Incremental update to aggregated information for data warehouses over Internet

Chan, Miranda; Leong, Hong Va; Si, Antonio

doi:10.1145/355068.355316

Cited by 10 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For incremental load, the observation timestamp for the previous delta load has to be maintained in an ETL metadata model. The data warehouse refresh is performed via batch cycles, so that the degree of information in a data warehouse is "predictable" [4]. In real world situations, there is a tendency by ETL programmers to design delta stored procedures to run 'stand-alone' -that is, without taking advantage of an ETL metadata model.…”

Section: Using Metadata In Load Processmentioning

confidence: 99%

An ETL Metadata Model for Data Warehousing

Rahman

Marz

Akhter

2012

CIT

View full text Add to dashboard Cite

Metadata is essential for understanding information stored in data warehouses. It helps increase levels of adoption and usage of data warehouse data by knowledge workers and decision makers. A metadata model is important to the implementation of a data warehouse; the lack of a metadata model can lead to quality concerns about the data warehouse. A highly successful data warehouse implementation depends on consistent metadata. This article proposes adoption of an ETL (extracttransform-load) metadata model for the data warehouse that makes subject area refreshes metadata-driven, loads observation timestamps and other useful parameters, and minimizes consumption of database systems resources. The ETL metadata model provides developers with a set of ETL development tools and delivers a user-friendly batch cycle refresh monitoring tool for the production support team.

show abstract

Section: Using Metadata In Load Processmentioning

confidence: 99%

An ETL Metadata Model for Data Warehousing

Rahman

Marz

Akhter

2012

CIT

View full text Add to dashboard Cite

show abstract

“…Therefore, the feature data is piped to the component returning similar history data (4) for the purpose of determining the subset of archived data most similar to the on-going data. Archived data is retrieved (5) and returned to the model (6 & 7). After that, the model gives its analysis based on on-going measurements and knowledge from archived measurements.…”

Section: Architecture and Operation Of Smart Archivementioning

confidence: 99%

“…The work approaches the problem from a data warehouse (DW) perspective and does not describe a framework for implementing DM applications. Other DW-centric studies are presented in [4], [5]. Architectures for processing data streams or data feeds have been developed in [6], [7], [8].…”

Section: Introductionmentioning

confidence: 99%

Smart archive: a component-based data mining application framework

Laurinen

Tuovinen

Röning

2005

5th International Conference on Intelligent Systems Design and Applications (ISDA'05)

View full text Add to dashboard Cite

Implementation of data mining applications is a challenging and complicated task, and the applications are often built from scratch. In this paper, a component-based application framework, called Smart Archive (SA) designed for implementing data mining applications, is presented. SA provides functionality common to most data mining applications and components for utilizing history information. Using SA, it is possible to build high-quality applications with shorter development times by configuring the framework to process application-specific data. The architecture, the components, the implementation and the design principles of the framework are presented. The advantages of a framework-based implementation are demonstrated by presenting a case study which compares the framework approach to implementing a real-world application with the option of building an equivalent application from scratch. In conclusion, the paper presents a lucid framework for creating data mining applications and illustrates the importance and advantages of using the presented approach.

show abstract

“…The view materialization problem is also considered and incremental update mechanism has been developed. This approach has been important in new data warehousing applications, especially when the data warehouse can be Web‐enabled (Chan, Leong, & Si, 2000), and even XML‐based. It is not difficult to associate the probability with an entity inside an XML document in a similar way, as it is associated with a tuple.…”

Section: Retrieval Modelsmentioning

confidence: 99%

A survey in indexing and searching XML documents

Luk

Leong

Dillon

et al. 2002

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

XML holds the promise to yield (1) a more precise search by providing additional information in the elements, (2) a better integrated search of documents from heterogeneous sources, (3) a powerful search paradigm using structural as well as content specifications, and (4) data and information exchange to share resources and to support cooperative search. We survey several indexing techniques for XML documents, grouping them into flatfile, semistructured, and structured indexing paradigms. Searching techniques and supporting techniques for searching are reviewed, including full text search and multistage search. Because searching XML documents can be very flexible, various search result presentations are discussed, as well as database and information retrieval system integration and XML query languages. We also survey various retrieval models, examining how they would be used or extended for retrieving XML documents. To conclude the article, we discuss various open issues that XML poses with respect to information retrieval and database research. IntroductionAn Internet search engine (e.g., Altavista or Infoseek) returns thousands of so-called matched documents from a single query, some of which are relevant and others irrelevant to the query. End users typically have problems with organizing and digesting such vast quantities of information, in which much (i.e., 75% as pointed out by Selberg and Etzioni, 1997) of the information retrieved is likely to be irrelevant. XML holds the promise that searching can be done more precisely because structural, self-describing information and meta-data (e.g., RDF) is available, to allow for context-based and/or category-based search. XML also holds the promise to model heterogeneous data, generated from databases (DBs) or from word processors, thereby enabling search engines to locate and process heterogeneous documents or records.An XML document consists of a set of elements, which are hierarchically structured, as defined by the user. Each element has a name (e.g., p for paragraph), which is defined by the user. Data of an element (say, p) can be stored inside the element delimited by its start tag (i.e., ͗p͘) and its end tag (i.e., ͗/p͘), or it can be stored as values in its attribute (e.g., ͗p idϭ "1"͘). Certain attribute value types are reserved for referencing (e.g., IDREF). An XML element is accessed typically using the XPath language. Child elements and their parent element are separated by a slash. For example, the XPath /header/author/first accesses the first element from the root element header, and then the author element.It is possible to use other mark-up languages (e.g., HTML) or proprietary formats but XML appears to be suitable for a wide variety of information retrieval (IR) tasks, specific enough to reduce modeling complexity and open enough for easy and rapid adoption. A major advantage of XML over HTML is that users can define their own tags. Tag names are typically chosen to incorporate some relationship to the semantics of the contents or the type of co...

show abstract

Incremental update to aggregated information for data warehouses over Internet

Cited by 10 publications

References 10 publications

An ETL Metadata Model for Data Warehousing

An ETL Metadata Model for Data Warehousing

Smart archive: a component-based data mining application framework

A survey in indexing and searching XML documents

Contact Info

Product

Resources

About