Data Quality Dimensions, Metrics, and Improvement Techniques

Abstract

Achieving high level of data quality is considered one of the most important assets for any small, medium and large size organizations. Data quality is the main hype for both practitioners and researchers who deal with traditional or big data. The level of data quality is measured through several quality dimensions. High percentage of the current studies focus on assessing and applying data quality on traditional data. As we are in the era of big data, the attention should be paid to the tremendous volume of generated and processed data in which 80% of all the generated data is unstructured. However, the initiatives for creating big data quality evaluation models are still under development. This paper investigates the data quality dimensions that are mostly used in both traditional and big data to figure out the metrics and techniques that are used to measure and handle each dimension. A complete definition for each traditional and big data quality dimension, metrics and handling techniques are presented in this paper. Many data quality dimensions can be applied to both traditional and big data, while few number of quality dimensions are either applied to traditional data or big data. Few number of data quality metrics and barely handling techniques are presented in the current works.

Knowledge Management as a Mediating Factor in the Relationship between Data Storytelling and Organizational Innovation

“…Rehm and Goel, 2015 Storey and Kelly (2002 Singh and Masuku, 2014 ) . (Gabr, 2021;Pipino et al, 2002) G _ r Z ? (Evergreen and Emery, 2016…”

Evaluation of the Limit of Detection in Network Dataset Quality Assessment with PerQoDA

“…Evaluating the dataset quality is challenging and must be done prior to any data modeling. While there are metrics that evaluate some important properties of a dataset (accuracy, completeness, consistency, timeliness, and others), these metrics often overlap [16]. Also, these metrics are more focused on the quality of data, and there is a lack of complete and proven methodologies for assessing the quality of datasets from a general perspective [30].…”

Tracking The Sensitivity of The Learning Models Toward Exact and Near Duplicates

“…Many data quality dimensions have been addressed through the literature [1]- [5], Data Duplication has been considered as one of the most intriguing dimensions. Data duplication is defined as multiple representation of the same real world object or a measure of undesirable duplicates within a certain field, record or dataset [6]. The duplication can be found in two different types, the Deterministic and the Probabilistic duplications [7], [8].…”