Enhancing the missing data imputation of primary substation load demand records

Borges, Cruz E.; Kamara-Esteban, Oihane; Castillo-Calzadilla, Tony; Martin, Cristina; Alonso-Vicario, Ainhoa

doi:10.1016/j.segan.2020.100369

Cited by 12 publications

(12 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the lack of imputation methods for energy time series -to the best knowledge of the authors -, we include imputation methods for power time series and time series in general despite their disadvantage of not utilizing energy data. Methods requiring additional data or information such as weather data [9], [13] or validated reference days [19] and methods designed for multivariate time series only [14], [20], [22] are discarded due to their lack of comparability. Furthermore, during the evaluation, the method in [15] is excluded due to its excessive run-time.…”

Section: B Benchmark Methodsmentioning

confidence: 99%

“…In [21], the time series measured by smart meters in a factory are used to impute missing values in other time series from smart meters located in the same factory with clustering and k-nearest neighbors. In [22], the imputation of substation data is formulated as a forecasting problem. The forecast uses the collected power data of nearby substations as well as weather data, which often has an impact on power consumption and generation.…”

Section: Introduction and State Of The Artmentioning

confidence: 99%

See 1 more Smart Citation

Data-Driven Copy-Paste Imputation for Energy Time Series

Weber

Turowski

Çakmak

et al. 2021

IEEE Trans. Smart Grid

View full text Add to dashboard Cite

A cornerstone of the worldwide transition to smart grids are smart meters. Smart meters typically collect and provide energy time series that are vital for various applications, such as grid simulations, fault-detection, load forecasting, load analysis, and load management. Unfortunately, these time series are often characterized by missing values that must be handled before the data can be used. A common approach to handle missing values in time series is imputation. However, existing imputation methods are designed for power time series and do not take into account the total energy of gaps, resulting in jumps or constant shifts when imputing energy time series. In order to overcome these issues, the present paper introduces the new Copy-Paste Imputation (CPI) method for energy time series. The CPI method copies data blocks with similar characteristics and pastes them into gaps of the time series while preserving the total energy of each gap. The new method is evaluated on a real-world dataset that contains six shares of artificially inserted missing values between 1 and 30%. It outperforms the three benchmark imputation methods selected for comparison. The comparison furthermore shows that the CPI method uses matching patterns and preserves the total energy of each gap while requiring only a moderate run-time.

show abstract

Section: B Benchmark Methodsmentioning

confidence: 99%

Section: Introduction and State Of The Artmentioning

confidence: 99%

Data-Driven Copy-Paste Imputation for Energy Time Series

Weber

Turowski

Çakmak

et al. 2021

IEEE Trans. Smart Grid

View full text Add to dashboard Cite

show abstract

“…Therefore, a statistical imputation method is required. Statistical imputation is a method by which the distribution of the time at which the month and day of the time to be replaced is the same, and the missing value can be replaced by the median value of the corresponding time [25]. For example, in the case of a person living alone, because they go to work around 8 a.m. on weekdays, there is no special energy consumption.…”

Section: The Methods For Hybrid-imputation Model For Missing Datamentioning

confidence: 99%

Empirical Analysis of the Impact of COVID-19 Social Distancing on Residential Electricity Consumption Based on Demographic Characteristics and Load Shape

et al. 2021

View full text Add to dashboard Cite

Since January 2020, the COVID-19 pandemic has been impacting various aspects of people’s daily lives and the economy. The first case of COVID-19 in South Korea was identified on 20 January 2020. The Korean government implemented the first social distancing measures in the first week of March 2020. As a result, energy consumption in the industrial, commercial and educational sectors decreased. On the other hand, residential energy consumption increased as telecommuting work and remote online classes were encouraged. However, the impact of social distancing on residential energy consumption in Korea has not been systematically analyzed. This study attempts to analyze the impact of social distancing implemented as a result of COVID-19 on residential energy consumption with time-varying reproduction numbers of COVID-19. A two-way fixed effect model and demographic characteristics are used to account for the heterogeneity. The changes in household energy consumption by load shape group are also analyzed with the household energy consumption model. There some are key results of COVID-19 impact on household energy consumption. Based on the hourly smart meter data, an average increase of 0.3% in the hourly average energy consumption is caused by a unit increase in the time-varying reproduction number of COVID-19. For each income, mid-income groups show less impact on energy consumption compared to both low-income and high-income groups. In each family member, as the number of family members increases, the change in electricity consumption affected by social distancing tends to decrease. For area groups, large area consumers increase household energy consumption more than other area groups. Lastly, The COVID-19 impact on each load shape is influenced by their energy consumption patterns.

show abstract

“…Due to the lack of imputation methods for energy time series -to the best knowledge of the authors -, we consider imputation methods for power time series and time series in general. Methods requiring additional data such as weather data [9], [13] or validated reference days [19] and methods designed for multivariate time series only [14], [20], [21] are discarded due to their lack of comparability. Furthermore, during the evaluation, the method in [15] is excluded due to its excessive run-time.…”

Section: B Benchmark Methodsmentioning

confidence: 99%

“…In [20], a method for imputation, denoising, and outlier removal based on Principal Component Pursuit is introduced that utilizes the spatial correlations of load profiles of adjacent substations. Similarly, in [21] the imputation of substation data is formulated as a forecasting problem, utilizing collected data of nearby substations as well as weather data, which often has an impact on power consumption and generation.…”

Section: Introduction and State Of The Artmentioning

confidence: 99%

Data-Driven Copy-Paste Imputation for Energy Time Series

Weber,

Turowski,

Çakmak

et al. 2021

Preprint

View full text Add to dashboard Cite

A cornerstone of the worldwide transition to smart grids are smart meters. Smart meters typically collect and provide energy time series that are vital for various applications, such as grid simulations, fault-detection, load forecasting, load analysis, and load management. Unfortunately, these time series are often characterized by missing values that must be handled before the data can be used. A common approach to handle missing values in time series is imputation. However, existing imputation methods are designed for power time series and do not take into account the total energy of gaps, resulting in jumps or constant shifts when imputing energy time series. In order to overcome these issues, the present paper introduces the new Copy-Paste Imputation (CPI) method for energy time series. The CPI method copies data blocks with similar properties and pastes them into gaps of the time series while preserving the total energy of each gap. The new method is evaluated on a real-world dataset that contains six shares of artificially inserted missing values between 1 and 30%. It outperforms by far the three benchmark imputation methods selected for comparison. The comparison furthermore shows that the CPI method uses matching patterns and preserves the total energy of each gap while requiring only a moderate run-time.

show abstract

Enhancing the missing data imputation of primary substation load demand records

Cited by 12 publications

References 47 publications

Data-Driven Copy-Paste Imputation for Energy Time Series

Data-Driven Copy-Paste Imputation for Energy Time Series

Empirical Analysis of the Impact of COVID-19 Social Distancing on Residential Electricity Consumption Based on Demographic Characteristics and Load Shape

Data-Driven Copy-Paste Imputation for Energy Time Series

Contact Info

Product

Resources

About