Influence of Data Distribution in Missing Data Imputation

Santos, Miriam Seoane; Soares, J.; Abreu, Pedro Henriques; Araújo, Hélder; Domingues, Inês

doi:10.1007/978-3-319-59758-4_33

Cited by 28 publications

(33 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Where the feature that contains missing values is used as a target, the remaining attributes are used as training data. After fitting the model, the missing values are identified as if they were class labels [43], [44]. The advantages of this method are it produces more accurate values and is available for both categorical and numeric variables; however, it is also more time-consuming [45].…”

Section: Data Pre-processingmentioning

confidence: 99%

Predicting Cervical Cancer using Machine Learning Methods

Alsmariy¹,

Healy²,

Abdelhafez³

2020

IJACSA

View full text Add to dashboard Cite

In almost all countries, precautionary measures are less expensive than medical treatment. The early detection of any disease gives a patient better chances of successful treatment than disease discovery at an advanced stage of its development. If we do not know how to treat patients, any treatment we can provide would be useful and would provide a more comfortable life. Cervical cancer is one such disease, considered to be fourth among the most common types of cancer in women around the world. There are many factors that increase the risk of cervical cancer, such as age and use of hormonal contraceptives. Early detection of cervical cancer helps to raise recovery rates and reduce death rates. This paper aims to use machine learning algorithms to find a model capable of diagnosing cervical cancer with high accuracy and sensitivity. The cervical cancer risk factor dataset from the University of California at Irvine (UCI) was used to construct the classification model through a voting method that combines three classifiers: Decision tree, logistic regression and random forest. The synthetic minority oversampling technique (SMOTE) was used to solve the problem of imbalance dataset and, together with the principal component analysis (PCA) technique, to reduce dimensions that do not affect model accuracy. Then, stratified 10-fold cross-validation technique was used to prevent the overfitting problem. This dataset contains four target variables-Hinselmann, Schiller, Cytology, and Biopsy-with 32 risk factors. We found that using the voting classifier, SMOTE and PCA techniques helped raise the accuracy, sensitivity, and area under the Receiver Operating Characteristic curve (ROC_AUC) of the predictive models created for each of the four target variables to higher rates. In the SMOTE-voting model, accuracy, sensitivity and PPA ratios improved by 0.93 % to 5.13 %, 39.26 % to 46.97 % and 2 % to 29 %, respectively for all target variables. Moreover, using PCA technology reduced computational processing time and increasing model efficiency. Finally, after comparing our results with several previous studies, it was found that our models were able to diagnose cervical cancer more efficiently according to certain evaluation measures.

show abstract

Section: Data Pre-processingmentioning

confidence: 99%

Predicting Cervical Cancer using Machine Learning Methods

Alsmariy¹,

Healy²,

Abdelhafez³

2020

IJACSA

View full text Add to dashboard Cite

show abstract

“…The efficiency of imputation techniques is PAC concerned to get the real value in the data. Pearson Correlation Coefficient (r) and Root Mean-Squared Error (RMSE) are two measures for evaluation of PAC [24,25]. Pearson correlation coefficient provides a measure of the correlation between the value of the imputation results with the actual value.…”

Section: Imputation Performancementioning

confidence: 99%

“…Pearson correlation coefficient provides a measure of the correlation between the value of the imputation results with the actual value. An imputation technique is efficient when the correlation value close to 1 [24,25]. If x are the attribute values in the complete data and x are the attribute values in the incomplete data then the correlation coefficient is calculated by formula (8)…”

Section: Imputation Performancementioning

confidence: 99%

“…Root Mean Squared Error (RMSE) has been called the most benchmark for comparison the performance of prediction strategies by measuring the distinction between the imputation value and the real value of a given feature. In this case, a value closer to 0 results in a better imputation [24,25]. Fit to model is calculated using the RMSE so that the RMSE describes how closely the predicted value is related to the true value.…”

Section: Imputation Performancementioning

confidence: 99%

“…In previous studies, most of the data estimation methods were missing, imputing data regardless of the dependence between attributes [22]. The performance of imputation algorithm of the missing data is significantly influenced by correlation structure in the data [1,4,23], missing data mechanism, distribution of missing data [24,25], and the proportion of the missing data within the data [1].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Class Center-Based Firefly Algorithm for Handling Missing Data

Nugroho

Utama

Surendro

2020

Preprint

View full text Add to dashboard Cite

Estimating missing data in a dataset is a significant advance during the data cleaning stage. Improper data handling can make inaccurate results when conducting data analysis. Most of the research about missing data estimation is irrespective of the correlation between attributes. However, an adaptive search procedure helps find the estimates of the missing data when correlations between attributes are considered in the process. Firefly Algorithm (FA) implements an adaptive search procedure in the imputation of the missing data by finding the estimated value that is closest to the value in other data known. Therefore, this study proposes a class center-based adaptive approach model for missing data by considering the attribute correlation in the imputation process (C3-FA). Based on the experiment, the general result find that the class center-based firefly algorithm is an efficient technique for getting the actual value in handling the missing data. This can be seen on the value of Pearson correlation coefficient (r) that close to 1 and the root mean squared error (RMSE) value is generally closer to 0. In addition, the proposed method can maintain the true distribution of data values. This is indicated by the Kolmogorov–Smirnov test that value of DKS for most of the attributes in the dataset is generally closer to 0. Also, the results of the accuracy evaluation using three classifiers, showed that the proposed method produces good accuracy.

show abstract