Big Data Pre-processing: A Quality Framework

Taleb, Ikbal; Dssouli, Rachida; Serhani, Mohamed Adel

doi:10.1109/bigdatacongress.2015.35

Cited by 86 publications

(46 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The words are handled which gives an improvement in accuracy [10]. Big data cleansing using data auto-discovery of quality rules and some conditional function dependency.…”

Section: Related Work In Big Data Analytics and Processingmentioning

confidence: 99%

A Big Data Model for an Inference Engine of a Production Company

S¹,

Tjprc²

2018

IJMPERD

View full text Add to dashboard Cite

show abstract

“…The words are handled which gives an improvement in accuracy [10]. Big data cleansing using data auto-discovery of quality rules and some conditional function dependency.…”

Section: Related Work In Big Data Analytics and Processingmentioning

confidence: 99%

A Big Data Model for an Inference Engine of a Production Company

S¹,

Tjprc²

2018

IJMPERD

View full text Add to dashboard Cite

show abstract

“…Taleb has proposed quality evaluation to the pre-processing phase of a big data system [9]. The main differences to our work are focus on medical data, and focusing on data quality with a data cleansing algorithm.…”

Section: Comparison To Literaturementioning

confidence: 99%

“…timeliness, author reputation). A quality framework has been developed for supporting data quality profile selection and adaptation [9]. Particularly, the framework has been implemented for pre-processing of medical data.…”

Section: Literature Reviewmentioning

confidence: 99%

Quality management architecture for social media data

Pääkkönen

Jokitulppo²

2017

J Big Data

View full text Add to dashboard Cite

“…Data preprocessing is an important procedure to increase the data quality of big data. Five tasks of data preprocessing of big data application [7], [8] discusses as follows: 1) Data cleansing: For assuring data quality, at first data cleansing need identify the data defects and problems. Then, according the types of data defects and problems, fill in missing values, smooth noisy data, recheck or remove abnormal data, and adjust the incomplete or inconsistent data.…”

Section: Major Tasks Of Data Preprocessingmentioning

confidence: 99%

An Iterative and Incremental Data Quality Improvement Procedure for Reducing the Risk of Big Data Project

Lai¹

2017

JSW

View full text Add to dashboard Cite

Big data applications can enhance the market competitive advantages of enterprises and organizations and can improve people's quality of life. However, by the impact of many factors, failure rate of big data project is higher than the IT project. In order to reduce the risk of failure, big data projects must overcome a serial of challenges. Ambiguous requirements, poor data quality, and lacking changeability and extensity will directly affect the results of big data analytics. And even cause the wrong decision, inaccurate prediction and improper planning to make the big data projects with potential high risk. For this, this paper migrates iterative and incremental development (IID) features to the data preprocessing, and draws up the iterative and incremental data quality improvement (IIDQI) procedure. IIDQI procedure applies data preprocessing task frame to repeatedly detect and identify the defects of data quality, and incrementally strengthen big data quality and control the factors of failure risk. Iterative inspection activities can effectively enhance data quality, intercommunication efficiency, and precision requirement and objective to reduce the risk of big data project failure.

show abstract

Big Data Pre-processing: A Quality Framework

Cited by 86 publications

References 30 publications

A Big Data Model for an Inference Engine of a Production Company

A Big Data Model for an Inference Engine of a Production Company

Quality management architecture for social media data

An Iterative and Incremental Data Quality Improvement Procedure for Reducing the Risk of Big Data Project

Contact Info

Product

Resources

About