Analysis of Data Quality and Performance Issues in Data Warehousing and Business Intelligence

Abstract

A Data Warehouse is an integral part of those enterprises which want to have a clear business insights from customer and operational data. It includes collection of technologies aimed at enabling the knowledge worker (executive, manager, analyst) to make better and faster decisions. It is expected to present the right information in the right place at the right time with the right cost in order to support the right decision. Over the years ,the practice of Data warehousing proved that the traditional online Transaction Processing (OLTP) systems are not fully appropriate for decision support. From the survey and evaluation of the literature related to Data Warehouse and with consultation and feedback of the data warehouse practitioners working in renowned IT giants ,it has been observed that the fundamental problems arise in populating a warehouse with quality data. . This paper mainly focuses on the study of the issues that hinder the data quality and performance of the Data warehouse and some of the means that may be opted to realize a better performance with respect to accuracy and quality to meet the challenging and dynamic needs of the corporate world.

Data Integration in Big Data Environment

“…After loading the data, it"s common to run audit reports so that there can be review of the results of the merged databases and systems to make sure the new data hasn"t caused any errors. The following diagram describes the general steps in traditional data warehousing process [10]. The followings are the some of the QOX ( Quality of Service, Quality of Data, Quality of Information etc.,) metrics in data integration [21], [22].…”

“…Also this paper is very helpful to find the open problems of data integration in Big Data Environment and the problems are, Integrating Crowd sourcing data, Integrating Data from data markets and providing an exploration tool for data sources. Nikhil Debbarma et al, [10] made study on the issues related to Data Warehouse in the quality perspective. Data quality plays an important role in Data Warehousing and Data Warehouse plays a vital role to extract right information from the right place at the right time with the right cost, in order to take right decision.…”

Analysis of Data Quality Issues in Big Data Migration Process

“…Nikhil Debbarma, Gautam Nath, Hillol Das (2013) -They discussed quality and performance issues in data warehouse [14] .…”

“…It creates malfunctioning, operational inefficiency, inefficient business analysis, unavailability of timely data. And also it affects customer satisfaction, reputation or even strategic decisions [14] . Implementation of Data Warehouse is exact solution to complex business intelligent applications.…”

Data Cleaning In Data Warehouse: A Survey of Data Pre-processing Techniques and Tools

“…Business Intelligence based decisions are based on clean data (Debbarma, Nath et al 2013). So before storage, data needs to be preprocessed, in order to improve productivity of ETL (Extraction, Transformation, and Loading) process.…”