Missing value imputation: a review and analysis of the literature (2006–2017)

Lin, Wei; Tsai, Chih‐Fong

doi:10.1007/s10462-019-09709-4

Cited by 378 publications

(229 citation statements)

References 114 publications

Supporting

Mentioning

188

Contrasting

Unclassified

Order By: Relevance

“…While there are a variety of reasons that data are missing, it is generally agreed that missing data disrupt analysis and practice, and so missing values are often imputed or not factored in at all. 27,28 An ideal data set would be valid, reliable, complete, and relevant. 29 In this study, we examined the role played by these last two characteristics, completeness and relevance, in understanding the value of SDH variables and their relevant contribution during care transition planning to avoid readmission.…”

Section: Discussionmentioning

confidence: 99%

Toward Understanding the Value of Missing Social Determinants of Health Data in Care Transition Planning

2020

View full text Add to dashboard Cite

Background Social determinants of health play an important role in the likelihood of readmission and therefore should be considered in care transition planning. Unfortunately, some social determinants that can be of value to care transition planners are missing in the electronic health record. Rather than trying to understand the value of data that are missing, decision makers often exclude these data. This exclusion can lead to failure to design appropriate care transition programs, leading to readmissions. Objectives This article examines the value of missing social determinants data to emergency department (ED) revisits, and subsequent readmissions. Methods A deidentified data set of 123,697 people (18+ years), with at least one ED visit in 2017 at the University of Alabama at Birmingham Medical Center was used. The dependent variable was all-cause 30-day revisits (yes/no), while the independent variables were missing/nonmissing status of the social determinants of health measures. Logistic regression was used to test the relationship between likelihood of revisits and social determinants of health variables. Moreover, relative weight analysis was used to identify relative importance of the independent variables. Results Twelve social determinants were found to be most often missing. Of those 12, only “lives with” (alone or with family/friends) had higher odds of ED revisits. However, relative logistic weight analysis suggested that “pain score” and “activities of daily living” (ADL) accounted for almost 50% of the relevance for ED revisits when compared among all 12 variables. Conclusion In the process of care transition planning, data that are documented are factored into the care transition plan. One of the most common challenges in health services practice is to understand the value of missing data in effective program planning. This study suggests that the data that are not documented (i.e., missing) could play an important role in care transition planning as a mechanism to reduce ED revisits and eventual readmission rates.

show abstract

Section: Discussionmentioning

confidence: 99%

Toward Understanding the Value of Missing Social Determinants of Health Data in Care Transition Planning

2020

View full text Add to dashboard Cite

show abstract

“…According to [24][25][26], the methods to deal with incomplete data containing missing values can be classified into three categories, which are case deletion, learning without handling of missing values, and missing value imputation. In case deletion, which is the simplest method, the data with missing values are removed from the original incomplete dataset to make it become a complete dataset.…”

Section: Missing Value Imputationmentioning

confidence: 99%

The Feature Selection Effect on Missing Value Imputation of Medical Datasets

et al. 2020

Self Cite

View full text Add to dashboard Cite

In practice, many medical domain datasets are incomplete, containing a proportion of incomplete data with missing attribute values. Missing value imputation can be performed to solve the problem of incomplete datasets. To impute missing values, some of the observed data (i.e., complete data) are generally used as the reference or training set, and then the relevant statistical and machine learning techniques are employed to produce estimations to replace the missing values. Since the collected dataset usually contains a certain number of feature dimensions, it is useful to perform feature selection for better pattern recognition. Therefore, the aim of this paper is to examine the effect of performing feature selection on missing value imputation of medical datasets. Experiments are carried out on five different medical domain datasets containing various feature dimensions. In addition, three different types of feature selection methods and imputation techniques are employed for comparison. The results show that combining feature selection and imputation is a better choice for many medical datasets. However, the feature selection algorithm should be carefully chosen in order to produce the best result. Particularly, the genetic algorithm and information gain models are suitable for lower dimensional datasets, whereas the decision tree model is a better choice for higher dimensional datasets.

show abstract

“…Tables IIa -IIc contain complete records of attributes V 1 , V 2 and V 3 , separately, which can be used to calculate the basic probability in (3). Tables IId and IIe contains complete records for the combination of V 1 and V 2 and the combination of all three input variables, which can be further utilized to obtain the joint basic probability in (4). Note that, there is no requirement for a complete dataset, e.g.…”

Section: Datasetmentioning

confidence: 99%

“…However, when missing values depend on hypothetical values or some other variables' values, single imputation methods, such as mean or median substitution, and last observation carried forward (LOCF), have been frequently used but not recommended, due to the problem of biased distribution and increased noise. In order to deal with such issues, a more commonly used practice of multiple imputation was initially proposed by [2], which has been widely promoted and is now the basic solution method for incomplete dataset problems, see [3], [4] and among others. A variety of multiple imputation methods solve this problem by randomly drawing multiple imputations from the imputation distribution and also by introducing an additional error variance to each imputation [5].…”

Section: Introductionmentioning

confidence: 99%

Maximum Likelihood Evidential Reasoning-Based Hierarchical Inference with Incomplete Data

Liu

Sachan

Yang

et al. 2019

2019 25th International Conference on Automation and Computing (ICAC)

View full text Add to dashboard Cite

Data mining requires a pre-processing task where data are prepared, cleaned, integrated, transformed, reduced and discretized to ensure data quality. Incomplete data are commonly encountered during data cleaning, which can have major impact on the conclusions that will be drawn from the data. In order to effectively carry out inferential modelling or decision making from incomplete independent variables or explanatory variables and consider different types of uncertainties, this paper adopts a data-driven inferential modelling approach, Maximum Likelihood Evidential Reasoning (MAKER) framework, which takes advantage of incomplete datasets without any imputation that may be required by other conventional machine learning methods. The MAKER framework reflects the plausibility of different values of missing data and expresses data-driven support for different values of missing data.

show abstract

Missing value imputation: a review and analysis of the literature (2006–2017)

Cited by 378 publications

References 114 publications

Toward Understanding the Value of Missing Social Determinants of Health Data in Care Transition Planning

Toward Understanding the Value of Missing Social Determinants of Health Data in Care Transition Planning

The Feature Selection Effect on Missing Value Imputation of Medical Datasets

Maximum Likelihood Evidential Reasoning-Based Hierarchical Inference with Incomplete Data

Contact Info

Product

Resources

About