Evaluation of Three Simple Imputation Methods for Enhancing Preprocessing of Data with Missing Values

Abstract

One of the important stages of data mining is preprocessing, where the data is prepared for different mining tasks. Often, the real-world data tends to be incomplete, noisy, and inconsistent. It is very common that the data are not obtainable for every observation of every variable. So the presence of missing variables is obvious in the data set. A most important task when preprocessing the data is, to fill in missing values, smooth out noise and correct inconsistencies. This paper presents the missing value problem in data mining and evaluates some of the methods generally used for missing value imputation. In this work, three simple missing value imputation methods are implemented namely (1) Constant substitution, (2) Mean attribute value substitution and (3) Random attribute value substitution.The performance of the three missing value imputation algorithms were measured with respect to different rate or different percentage of missing values in the data set by using some known clustering methods. To evaluate the performance, the standard WDBC data set has been used.

Improving Prediction Accuracy of “Central Line-Associated Blood Stream Infections” Using Data Mining Models

“…If such values are not calculated by using appropriate means or incomplete records are not removed, they can cause poor analytical results [6, 7]. A number of methods can be used to replace the missing values by using measures like mean and median [51]. …”

A Sketch-based Clustering Algorithm for Uncertain Data Streams

“…We will show the accuracy of our algorithm based on the evaluation standard of RandIndex [20]. For uncertain data set D (includes N objects), let T={T 1 , T 2 , …, T k } represent the original clusters, and C={C 1 , C 2 , …, C m } be the clusters produced by a clustering algorithm.…”

Data Cleaning In Data Warehouse: A Survey of Data Pre-processing Techniques and Tools

“…Three algorithms have been presented below, that compute missing values and their attributes (M) in dataset (D) (Somasundaram and Nedunchezhian 2011).…”

“…2 nd approach of mean attribute value substitution method is time consuming & expensive, but gives best results for missing values problem. 3 rd approach of random attribute value substitution method causes distortion in data distributions by assuming that all missing values are with the same value, however this method still manages to provide comparable results (Somasundaram and Nedunchezhian 2011). Weak point of these techniques is the need of strong model assumptions.…”

“…These algorithms fill the missing values and smooth out the noise. Three of those implemented algorithms are Constant Substitution, Mean attribute value substitution and Random attribute value substitution method (Somasundaram and Nedunchezhian 2011). These methods have been tested on the standard WDBC dataset and their performance has been compared on the basis of defined evaluation attributes.…”