Systematic Review of Using Machine Learning in Imputing Missing Values

Alabadla, Mustafa; Sidi, Fatimah; Ishak, Iskandar; Ibrahim, Hamidah; Affendey, Lilly Suriani; Ani, Zafienas Che; Jabar, Marzanah A.; Bukar, Umar Ali; Muda, Ahmad Sobri; Tharek, Anas; Omar, Noritah; Jaya, M. Izham

doi:10.1109/access.2022.3160841

Cited by 21 publications

(2 citation statements)

References 111 publications

(101 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Handling of abnormal data and missing values [98]: Each kind of data has its own actual meaning behind it. When the data value exceeds the normal range or is a meaningless expression, it needs to be adjusted or supplemented in a targeted way.…”

Section: Dataset and Preprocessingmentioning

confidence: 99%

Machine Learning Model for Hepatitis C Diagnosis Customized to Each Patient

Chen

2022

IEEE Access

View full text Add to dashboard Cite

Machine learning is now widely used in various fields, and it has made a big splash in the field of disease diagnosis. But traditional machine learning models are general-purpose, that is, one model is used to evaluate the health status of different patients. A general-purpose machine learning algorithm depends on a large amount of data and requires abundant computing power support, relies on the average level to describe the model performance, and cannot achieve optimal results on a specific problem. In this paper, we propose to train a unique model for each patient to improve the accuracy and ease of use of the model. The proposed approach to solving a problem in the paper is from three perspectives (1) targeted data processing, (2) model structure design: Passing in patient-related information into the model, and (3) hyperparameter tailored optimization. The preliminary experimental results show that using the custom model has advantages of high accuracy, high confidence, and low resource required to diagnose a patient. In the Hepatitis C dataset, over 99% accuracy and 94% recall were achieved using a smaller dataset (only 615 individuals' data) without knowledge of the relevant field. Traditional algorithms such as XGBoost or multi-algorithm ensemble could achieve less than 95% accuracy and only less than 70% recall. Out of a total of 56 patients, the custom model was able to identify 53 patients 20 more than traditional methods, bringing a new and efficient tool for future hepatitis C prevention and treatment efforts. INDEX TERMS Machine learning, custom model, hepatitis C, disease diagnosis, data augmentation, parameter optimization.

show abstract

Section: Dataset and Preprocessingmentioning

confidence: 99%

Machine Learning Model for Hepatitis C Diagnosis Customized to Each Patient

Chen

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…An imputation algorithm “opt.impute” was introduced in [ 11 ] to achieve the finest solutions to the missing data. Further, an extensive review was conducted on the imputation of missing data using ML which helps in understanding the limitations of ML imputation methods [ 12 ]. A framework was implemented to improvise the multivariate imputation by chained equations (MICE) in imputing the missing sensor data [ 13 ].…”

Section: Introductionmentioning

confidence: 99%

Machine Learning-Based Ensemble Classifiers for Anomaly Handling in Smart Home Energy Consumption Data

Prakash

Kumar

Moganti

et al. 2022

Sensors

View full text Add to dashboard Cite

Addressing data anomalies (e.g., garbage data, outliers, redundant data, and missing data) plays a vital role in performing accurate analytics (billing, forecasting, load profiling, etc.) on smart homes’ energy consumption data. From the literature, it has been identified that the data imputation with machine learning (ML)-based single-classifier approaches are used to address data quality issues. However, these approaches are not effective to address the hidden issues of smart home energy consumption data due to the presence of a variety of anomalies. Hence, this paper proposes ML-based ensemble classifiers using random forest (RF), support vector machine (SVM), decision tree (DT), naive Bayes, K-nearest neighbor, and neural networks to handle all the possible anomalies in smart home energy consumption data. The proposed approach initially identifies all anomalies and removes them, and then imputes this removed/missing information. The entire implementation consists of four parts. Part 1 presents anomaly detection and removal, part 2 presents data imputation, part 3 presents single-classifier approaches, and part 4 presents ensemble classifiers approaches. To assess the classifiers’ performance, various metrics, namely, accuracy, precision, recall/sensitivity, specificity, and F1 score are computed. From these metrics, it is identified that the ensemble classifier “RF+SVM+DT” has shown superior performance over the conventional single classifiers as well the other ensemble classifiers for anomaly handling.

show abstract

Distributed personalized imputation based on Gaussian mixture model for missing data

Chen,

Liu

2024

Neural Comput & Applic

View full text Add to dashboard Cite

Systematic Review of Using Machine Learning in Imputing Missing Values

Cited by 21 publications

References 111 publications

Machine Learning Model for Hepatitis C Diagnosis Customized to Each Patient

Machine Learning Model for Hepatitis C Diagnosis Customized to Each Patient

Machine Learning-Based Ensemble Classifiers for Anomaly Handling in Smart Home Energy Consumption Data

Distributed personalized imputation based on Gaussian mixture model for missing data

Contact Info

Product

Resources

About