Techniques for dealing with incomplete data: a tutorial and survey

Aste, Marco; Boninsegna, M.; Freno, Antonino; Trentin, Edmondo

doi:10.1007/s10044-014-0411-9

Cited by 38 publications

(25 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second approach for imputing missing values is the machine learning approach. The k-nearest neighbor algorithm can be used to estimate missing values by finding the most similar complete k-data points or patterns and use their values [9,22]. A lot of work has also been done to build more elegant machine learning algorithms like neural networks and decision trees for imputation [9,22,23].…”

Section: Prior Imputation Methodsmentioning

confidence: 99%

“…The k-nearest neighbor algorithm can be used to estimate missing values by finding the most similar complete k-data points or patterns and use their values [9,22]. A lot of work has also been done to build more elegant machine learning algorithms like neural networks and decision trees for imputation [9,22,23]. Another method taking account of more values to improve the variance estimation is expectation maximization (EM) which uses the statistical maximum likelihood of a missing value.…”

Section: Prior Imputation Methodsmentioning

confidence: 99%

“…Then, the Expectation step finds the conditional expectation of the missing values given the observed data and current estimated parameters. These expectation values are then substituted for the missing values [9,[20][21][22]. This approach continues until there is convergence in the parameter estimates.…”

Section: Prior Imputation Methodsmentioning

confidence: 99%

See 2 more Smart Citations

The effect of imputing missing clinical attribute values on training lung cancer survival prediction model performance

Barakat

Field

Ghose

et al. 2017

Health Inf Sci Syst

View full text Add to dashboard Cite

According to the estimations of the World Health Organization and the International Agency for Research in Cancer, lung cancer is the most common cause of death from cancer worldwide. The last few years have witnessed a rise in the attention given to the use of clinical decision support systems in medicine generally and in cancer in particular. These can predict patients' likelihood of survival based on analysis of and learning from previously treated patients. The datasets that are mined for developing clinical decision support functionality are often incomplete, which adversely impacts the quality of the models developed and the decision support offered. Imputing missing data using a statistical analysis approach is a common method to addressing the missing data problem. This work investigates the effect of imputation methods for missing data in preparing a training dataset for a Non-Small Cell Lung Cancer survival prediction model using several machine learning algorithms. The investigation includes an assessment of the effect of imputation algorithm error on performance prediction and also a comparison between using a smaller complete real dataset or a larger dataset with imputed data. Our results show that even when the proportion of records with some missing data is very high (> 80%) imputation can lead to prediction models with an AUC (0.68-0.72) comparable to those trained with complete data records.

show abstract

Section: Prior Imputation Methodsmentioning

confidence: 99%

Section: Prior Imputation Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The effect of imputing missing clinical attribute values on training lung cancer survival prediction model performance

Barakat

Field

Ghose

et al. 2017

Health Inf Sci Syst

View full text Add to dashboard Cite

show abstract

“…One of the most popular approaches to deal with missing values is to fill them with predicted values (imputation) ], to which this work belongs. The missing data can be managed by training models for various combinations of modalities and by selecting an convenient model for each combination [9,10] or by applying generic methods to combine all modalities in the presence of missing data, such as imputation of the missing data or modification of the fusion algorithm [11]. [12,13] proposed an imputation boosted collaborative filtering technique (IBCF).…”

Section: Related Workmentioning

confidence: 99%

A Novel Imputation-Boosted Technique to Overcome the Unrated Items Issue and Improving the Performance of Collaborative Filtering

2018

Dec. 14-15, 2017 Kuala Lumpur (Malaysia)

View full text Add to dashboard Cite

show abstract

“…MNAR estimation methods, such as selection and pattern-mixture analyses, jointly model observed and missing information. Importantly, these models rely on strong, unverifiable assumptions [14] . Sensitivity analysis is therefore required to evaluate changes in analytical outcomes given different MNAR scenarios.…”

Section: Incomplete Datamentioning

confidence: 99%

Quantitative Methods in Psychological Aging Research: A Mini-Review

Ghisletta

Aichele²

2017

Gerontology

View full text Add to dashboard Cite

As research on psychological aging moves forward, it is increasingly important to accurately assess longitudinal changes in psychological processes and to account for their (often complex) associations with sociodemographic, lifestyle, and health-related variables. Traditional statistical methods, though time tested and well documented, are not always satisfactory for meeting these aims. In this mini-review, we therefore focus the discussion on recent statistical advances that may be of benefit to researchers in psychological aging but that remain novel in our area of study. We first compare two methods for the treatment of incomplete data, a common problem in longitudinal research. We then discuss robust statistics, which address the question of what to do when critical assumptions of a standard statistical test are not met. Next, we discuss two approaches that are promising for accurately describing phenomena that do not unfold linearly over time: nonlinear mixed-effects models and (generalized) additive models. We conclude by discussing recursive partitioning methods, as these are particularly well suited for exploring complex relations among large sets of variables.

show abstract

Techniques for dealing with incomplete data: a tutorial and survey

Cited by 38 publications

References 89 publications

The effect of imputing missing clinical attribute values on training lung cancer survival prediction model performance

The effect of imputing missing clinical attribute values on training lung cancer survival prediction model performance

A Novel Imputation-Boosted Technique to Overcome the Unrated Items Issue and Improving the Performance of Collaborative Filtering

Quantitative Methods in Psychological Aging Research: A Mini-Review

Contact Info

Product

Resources

About