Mind the gap

Khayati, Mourad; Lerner, Alberto; Tymchenko, Zakhar; Cudré-Mauroux, Philippe

doi:10.14778/3377369.3377383

Cited by 54 publications

(16 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The imputation performance of each method was evaluated by measuring the MAE and RMSE of the imputed and original data. MAE and RMSE are defined as: (5) In addition, the proposed method was compared with the experimental results of existing methods, such as locf, nocb, nearest, linear, spline, mean, median, and knn. In the initial parameter-setting stage, the window size was set to five for the mean and median methods using the windows.…”

Section: Resultsmentioning

confidence: 99%

“…Missing values can lead to biased results and affect the performance of machine learning algorithms [1,3,4]. In particular, "blackouts" are extreme missing scenarios, in which all the sensors are quiet simultaneously, causing widespread and aligned missing blocks [5]. Until recently, few algorithms have imputed missing blocks with high accuracy in blackouts [5].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Imputation Method Using Local Linear Regression Based on Bidirectional k-nearest-components

Lee

2023

JICCE

View full text Add to dashboard Cite

This paper proposes an imputation method using a bidirectional k-nearest components search based local linear regression method. The bidirectional k-nearest-components search method selects components in the dynamic range from the missing points. Unlike the existing methods, which use a fixed-size window, the proposed method can flexibly select adjacent components in an imputation problem. The weight values assigned to the components around the missing points are calculated using local linear regression. The local linear regression method is free from the rank problem in a matrix of dependent variables. In addition, it can calculate the weight values that reflect the data flow in a specific environment, such as a blackout. The original missing values were estimated from a linear combination of the components and their weights. Finally, the estimated value imputes the missing values. In the experimental results, the proposed method outperformed the existing methods when the error between the original data and imputation data was measured using MAE and RMSE.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Imputation Method Using Local Linear Regression Based on Bidirectional k-nearest-components

Lee

2023

JICCE

View full text Add to dashboard Cite

show abstract

“…Moreover, these methods do not contend well with high data sparsity. In addition, there exist deep learning studies for imputating multi-variate time series data [10], [11], [13], [17], [36], [44], [56]. For instance, Che et al [13] incorporate masking and time-lag mechanisms into a vanilla GRU and impute nulls based on a weighted combination of the last observation and a global mean.…”

Section: Bisimmentioning

confidence: 99%

Data Imputation for Sparse Radio Maps in Indoor Positioning (Extended Version)

Li¹,

Li²,

Chan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Indoor location-based services rely on the availability of sufficiently accurate positioning in indoor spaces. A popular approach to positioning relies on so-called radio maps that contain pairs of a vector of Wi-Fi signal strength indicator values (RSSIs), called a fingerprint, and a location label, called a reference point (RP), in which the fingerprint was observed. The positioning accuracy depends on the quality of the radio maps and their fingerprints. Radio maps are often sparse, with many pairs containing vectors missing many RSSIs as well as RPs. Aiming to improve positioning accuracy, we present a complete set of techniques to impute such missing values in radio maps. We differentiate two types of missing RSSIs: missing not at random (MNAR) and missing at random (MAR). Specifically, we design a framework encompassing a missing RSSI differentiator followed by a data imputer for missing values. The differentiator identifies MARs and MNARs via clustering-based fingerprint analysis. Missing RSSIs and RPs are then imputed jointly by means of a novel encoder-decoder architecture that leverages temporal dependencies in data collection as well as correlations among fingerprints and RPs. A time-lag mechanism is used to consider the aging of data, and a sparsity-friendly attention mechanism is used to focus attention score calculation on observed data. Extensive experiments with real data from two buildings show that our proposal outperforms the alternatives with significant advantages in terms of imputation accuracy and indoor positioning accuracy.

show abstract

“…Notably, their study also evaluated performance in downstream Machine Learning (ML) tasks, finding that the imputation rendered a 10–20% performance increase. Khayati et al [ 20 ] focused on sensor time series imputation, comparing 16 recovery algorithms on six public and two synthetic datasets, including block missings, which are more reflective of WSN data characteristics. Their findings suggested that the optimal recovery method often depends on dataset-specific characteristics.…”

Section: Introductionmentioning

confidence: 99%

Missing Value Imputation of Wireless Sensor Data for Environmental Monitoring

Decorte,

Mortier,

Lembrechts

et al. 2024

Sensors

View full text Add to dashboard Cite

Over the past few years, the scale of sensor networks has greatly expanded. This generates extended spatiotemporal datasets, which form a crucial information resource in numerous fields, ranging from sports and healthcare to environmental science and surveillance. Unfortunately, these datasets often contain missing values due to systematic or inadvertent sensor misoperation. This incompleteness hampers the subsequent data analysis, yet addressing these missing observations forms a challenging problem. This is especially the case when both the temporal correlation of timestamps within a single sensor and the spatial correlation between sensors are important. Here, we apply and evaluate 12 imputation methods to complete the missing values in a dataset originating from large-scale environmental monitoring. As part of a large citizen science project, IoT-based microclimate sensors were deployed for six months in 4400 gardens across the region of Flanders, generating 15-min recordings of temperature and soil moisture. Methods based on spatial recovery as well as time-based imputation were evaluated, including Spline Interpolation, MissForest, MICE, MCMC, M-RNN, BRITS, and others. The performance of these imputation methods was evaluated for different proportions of missing data (ranging from 10% to 50%), as well as a realistic missing value scenario. Techniques leveraging the spatial features of the data tend to outperform the time-based methods, with matrix completion techniques providing the best performance. Our results therefore provide a tool to maximize the benefit from costly, large-scale environmental monitoring efforts.

show abstract

Mind the gap

Cited by 54 publications

References 52 publications

Imputation Method Using Local Linear Regression Based on Bidirectional k-nearest-components

Imputation Method Using Local Linear Regression Based on Bidirectional k-nearest-components

Data Imputation for Sparse Radio Maps in Indoor Positioning (Extended Version)

Missing Value Imputation of Wireless Sensor Data for Environmental Monitoring

Contact Info

Product

Resources

About