Missing value imputation techniques depth survey and an imputation Algorithm to improve the efficiency of imputation

Thirukumaran, S.; Sumathi, A.

doi:10.1109/icoac.2012.6416805

Cited by 23 publications

(11 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During the data construction process, handling missing data plays a major role as quality of the data gets enhanced to the maximum extent. Various techniques employed to handle missing data include deletion of the records leading towards information loss, averaging the values may induce irregularities in the prediction sequences with lot of systematic differences [5]. However the probability of missing values for an attribute may depend on that particular attribute's values to a major extent [6].…”

Section: Related Workmentioning

confidence: 99%

Data Pre-Processing and Customized Onto-Graph Construction for Knowledge Extraction in Healthcare Domain of Semantic Web

Monika¹,

Raju²

2019

IJITEE

View full text Add to dashboard Cite

Present electronic world produces enormous amount of data every second in various formats, especially in healthcare units. To efficiently utilize the available data by representing it in the machine readable form, the concept of Semantic web stepped in progressing towards automated knowledge discovery process. In this paper, comprehensive pre-processing techniques have been proposed for preparing the raw data to be presentable in structured format so as to construct the onto-graph for selected features in a health care domain. Cluster based Missing Value Imputation Algorithm (CMVI) has been proposed to enhance the quality of the imputed data which is the most important step during data pre-processing. Missing values were randomly induced into the Pima Indian Diabetic dataset with the missing ratio of 1%, 3% and 5% for each attribute up to 50% of the attributes in the original diabetic dataset. The experimental observations reveal that the quality of the pre-processed data is better compared to raw, unprocessed data in terms of imputation accuracy measured against coefficient of determination (R2 ), Index of agreement (d2 ) and Root Mean Square Error (RMSE).Documented results proved that the proposed techniques are comparatively superior than the traditional approaches with increased R2 & d2 and decreased RMSE scores. Further, importance of knowledge graph and various ontological representation types are discussed in short as construction of .owl file is the first step towards automation in semantic web.

show abstract

Section: Related Workmentioning

confidence: 99%

Data Pre-Processing and Customized Onto-Graph Construction for Knowledge Extraction in Healthcare Domain of Semantic Web

Monika¹,

Raju²

2019

IJITEE

View full text Add to dashboard Cite

show abstract

“…It is based on a regression technique that models relationships between dependent (i.e., the attribute with missing values) and independent variables (i.e., other attributes). Following that, Thirukmaran and Sumath [9] also explore the use of regression for imputing missing values. In addition, other techniques like replacement with means and expected maximization has also been experimented.…”

Section: Background and Related Workmentioning

confidence: 99%

Clustering data with the presence of missing values by ensemble approach

Pattanodom

Iam-On

Boongoen

2016

2016 Second Asian Conference on Defence Technology (ACDT)

View full text Add to dashboard Cite

The problem of missing values arise as one of the major difficulties in data mining and the downstreaming applications. In fact, most of the analytical techniques established in this field have been developed to handle a complete data set. Imputing or filling in missing values is generally regarded as a data preprocessing task, for which several methods has been introduced. These include a collection of statistical alternatives such as average and zero imputes, as well as learning-led models like nearest neighbors and regression. As for cluster analysis, various clustering algorithms, even k-means the most well-known, are hardly design to handle such a problem. This is also the case with cluster ensembles, where an improved decision is generated upon multiple results of clustering complete data. The paper presents a new framework that allows clustering incomplete data without the usual preprocessing step. Intuitively, different versions of the original data can be created by filling in those unknown values with arbitrary ones. This random selection is simple and efficient, while promotes the diversity within an ensemble, hence its quality. In particular, Binary clusterassociation matrix (BA) has been adopted to summarize ensemble information, from which k-means is exploited to derive the final clustering. The proposed model is evaluated against a number of benchmark imputation methods, over different datasets obtained from UCI repository. Based on the evaluation metric of cluster accuracy (CA), the findings suggest more accurate outcome is usually observed with the new framework. This motivates an application of the proposed approach to problems specific to Thai armed forces, such as identification of attacks that is presently in the spotlight for cyber security.

show abstract

“…In Table 2 we presented the approaches to fix the issues found in DQD phase from the review works [18], [22]- [24] and the research [25]. …”

Section: Clean Datamentioning

confidence: 99%

“…Several approaches exist to tackle the issues of data quality in outliers [22], noise [18], inconsistency [23], incompleteness [24], redundancy [26], [27], amount of data [28]- [30], heterogeneity [14], and timeliness [25]. Nevertheless the results to date not consider resolve the issues in ensemble.…”

Section: Conclusion and Future Researchmentioning

confidence: 99%

A Conceptual Framework for Data Quality in Knowledge Discovery Tasks (FDQ-KDT): A Proposal

Corrales¹,

Ledezma²,

Corrales³

2015

JCP

View full text Add to dashboard Cite

Large Volume of Data is growing because the organizations are continuously capturing the collective amount of data for better decision-making process. The most fundamental challenge is to explore the large volumes of data and extract useful knowledge for future actions through data mining and data science methodologies. Nevertheless these not tackle the issues in data quality clearly, leaving out relevant activities. We proposed a conceptual framework for data quality in knowledge discovery tasks based on CRISP-DM, SEMMA and Data Science, considering the issues of ESE Taxonomy.

show abstract

Missing value imputation techniques depth survey and an imputation Algorithm to improve the efficiency of imputation

Cited by 23 publications

References 25 publications

Data Pre-Processing and Customized Onto-Graph Construction for Knowledge Extraction in Healthcare Domain of Semantic Web

Data Pre-Processing and Customized Onto-Graph Construction for Knowledge Extraction in Healthcare Domain of Semantic Web

Clustering data with the presence of missing values by ensemble approach

A Conceptual Framework for Data Quality in Knowledge Discovery Tasks (FDQ-KDT): A Proposal

Contact Info

Product

Resources

About