A Novel Data Imputation Method (M-CBRI) for Industrial Analytic Applications

Abstract

Data analysis is mainly based on understanding and preprocessing the data coming from various sources for various applications. Missing values might play a critical role to reflect to characteristic of datasets; thus, imputation of missing values is a valuable process to not only handle reducing deviation but also avoid loss of data. There are different approaches to filling missing values. One of them is correlation-based imputation method. This approach is based on the high correlation between the parameters, these parameters are variables of linear equation, the linear equation enables to predict missing values. In this study, improvements were made to the correlation-based imputation method to predict missing values. The proposed method was performed on three various datasets which are related to the automotive industry. Missing values are handled in a manual process, and these values are picked randomly from the real data. After generating missing values, missing values are predicted using the correlation-based imputation method; furthermore, the margin of error between the estimated value and actual value was calculated. The results were compared to different methods which are arithmetic mean assignment, median value assignment, k- nearest neighbor assignment, and multivariate imputation by chained equations; consequently, much more successful results were obtained with the proposed method for three datasets.