Imputation of missing value using dynamic Bayesian network for multivariate time series data

Susanti, Steffi Pauli; Azizah, Fazat Nur

doi:10.1109/icodse.2017.8285864

Cited by 26 publications

(8 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this respect, temporal correlation is calculated using the time at which each data from a sensor were gathered. Linear interpolation [27], Last Observation Carried Forward (LOCF) [28], autoregressive model [29], and Support Vector Regression (SVR) [30] are commonly used to perform temporal imputation although they can also be used to perform other types of imputations. However, these methods do not handle long temporal gaps efficiently and have a tendency to increase bias.…”

Section: )mentioning

confidence: 99%

Long Gaps Missing IoT Sensors Time Series Data Imputation: A Bayesian Gaussian Approach

et al. 2022

View full text Add to dashboard Cite

Missing sensor data is a common problem associated with the Internet of Things (IoT) ecosystems, which affect the accuracy of the associated services such as adequate medical intervention for older adults living at home. This problem is caused by many factors, power down is one of them, communication failure and sensor failure are another two reasons. Multiple missing data imputation methods have been developed to solve this issue. However, irregular temporal missing data locations is challenging to handle, due to lack of knowledge of their occurrence probability and their random temporal location. In this paper, we propose a Bayesian Gaussian Process based imputation technique that accounts for temporal forcing to fill in the missing sensor data. Our approach; Bayesian Gaussian Process (BGaP); can impute missing data efficiently at any missing rate and for any temporal location using prior knowledge gathered on past observations. We illustrated how our approach performs using real data collected from sensors deployed in the residence of 10 older adults over a two-year period. Using our novel approach, we were able to impute all the missing data which allowed us to observe long-term behavior changes that we would not have been able to observe otherwise.INDEX TERMS Missing Sensor Data, Missing Data Imputation, Bayesian Gaussian Process, Long-term Older adults Behavior Monitoring.

show abstract

Section: )mentioning

confidence: 99%

Long Gaps Missing IoT Sensors Time Series Data Imputation: A Bayesian Gaussian Approach

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In addition, Myneni et al [21] presented a framework for correlated cluster-based imputation to improve the quality of data for data mining applications. Another work in literature [22] handled the missing data by using dynamic bayesian network and support vector regression algorithm is used for predicting the filling values.…”

Section: Related Workmentioning

confidence: 99%

Data-driven missing data imputation in cluster monitoring system based on deep neural network

Lin

Алам

et al. 2019

Appl Intell

View full text Add to dashboard Cite

Due to cluster instability, not in the cluster monitoring system. This paper focuses on the missing data imputation processing for the cluster monitoring application and proposes a new hybrid multiple imputation framework. This new imputation approach is different from the conventional multiple imputation technologies in the fact that it attempts to impute the missing data for an arbitrary missing pattern with a model-based and data-driven combination architecture. Essentially, the deep neural network, as the data model, extracts deep features from the data and deep features are further calculated then by a regression or data-driven strategies and used to create the estimation of missing data with the arbitrary missing pattern. This paper gives evidence that if we can train a deep neural network to construct the deep features of the data, imputation based on deep features is better than that directly on the original data. In the experiments, we compare the proposed method with other conventional multiple imputation approaches for varying missing data patterns, missing ratios, and different datasets including real cluster data. The result illustrates that when data encounters larger missing ratio and various missing patterns, the proposed algorithm has the ability to achieve more accurate and stable imputation performance. Keywords Missing data • Cluster monitoring • Deep belief networks • Multiple imputation 1 Introduction Clusters not only making an accurate decision but also monitoring with partial missing cluster data is the practical problem. Systems are becoming increasingly complex due to the large number of services and resources. The cluster monitoring system is what we use in a systematic way to process or analyze cluster data at a remote location under normal circumstances.

show abstract

“…Susanti and Azizah [12] proposed a data preprocess approach based on dynamic Bayesian network to handle the problem of missing values in order to maintaining the dependency relationships between attributes and features of data efficiently. Wei et al [13] analyzed three kinds of data missing in two separate clinical metabolomics datasets, i.e., Missing Completely At Random (MCAR), Missing At Random (MAR), and left-censored Missing Not At Random (MNAR), and developed a public-accessible web-tool for the application of handling missing data.…”

Section: A Data Preprocessingmentioning

confidence: 99%

Redundancy-Reducing and Holiday Speed Prediction Based on Highway Traffic Speed Data

et al. 2019

View full text Add to dashboard Cite

The accurate prediction of the highway traffic speed plays an important role in improving the production efficiency and the convenience of lives, and it is also one of the important issues in vehicle social big data analyses. Aiming at the problem of highway traffic speed data preprocessing, this paper proposes the algorithm of Redundant Data Reducing (RDR) that can greatly reduce the amount of data in model training of long short-term memory (LSTM) and improve the training speed under the condition that the influence on prediction accuracy is small and controllable. Aiming at the problem of low prediction accuracy for holiday traffic speeds due to small data volume, this paper proposes the Segment Prediction Algorithm (SPA) based on the speed features, which can effectively improve the prediction accuracy. The experimental results show that RDR can reduce the training data by up to 60% without notably affecting the prediction accuracy, while it improves the training speed by 60%. Compared with the current LSTM algorithm, the prediction accuracy of SPA improves significantly. INDEX TERMS Traffic speed data, redundancy reducing, holiday speed prediction, prediction accuracy.

show abstract

Imputation of missing value using dynamic Bayesian network for multivariate time series data

Cited by 26 publications

References 5 publications

Long Gaps Missing IoT Sensors Time Series Data Imputation: A Bayesian Gaussian Approach

Long Gaps Missing IoT Sensors Time Series Data Imputation: A Bayesian Gaussian Approach

Data-driven missing data imputation in cluster monitoring system based on deep neural network

Redundancy-Reducing and Holiday Speed Prediction Based on Highway Traffic Speed Data

Contact Info

Product

Resources

About