No abstract

No abstract

No abstract

Data warehouses are loaded with data from sources such as operational data bases. Failure of loading process or failure of any of the process such as extraction or transformation is expensive because of the non-availability of data for analysis. With the advent of e-commerce and many real time application analysis of data in real time becomes a norm and hence any misses while the data is being loaded into data warehouse needs to be handled in an efficient and optimized way. The techniques to handle failure of process to populate the data are very much important as the actual loading process. Alternative arrangement needs to be made for in case of failure so that processes of populating the data warehouse are done in time. This paper explores the various ways through which a failed process of populating the data warehouse could be resumed. Various resumption techniques are compared and a novel block based technique is proposed to improve one of the existing resumption techniques.

Refreshment anomalies occur in a data warehousing environment while performing Extract Transform and Load (ETL) to get the data for analysis from sources. There could be several reasons for the anomalies like not able to capture the delta on time, system time out, duplicate entries due to outer join operations and many more. Once anomalies are detected the compensation operation is executed to get the data that was missing into the data warehouse. In this work we would like to analyze scenario where it is necessary to perform incremental loads based on priority in an ongoing data warehouse maintenance work. The work proposes a novel approach to decide on when to perform ETL so that refreshment anomalies do not occur and to maintain integrity of data such that analytics queries always provide right information to the analyst. Two novelties have been discussed in this work one is to have a threshold before compensation of updates and two is while performing compensation updates prioritize the query with less freshness interval to have more time limits for the updates to be completed.