A practical approach to cluster validation in the energy sector

Bogensperger, Alexander; Fabel, Yann

doi:10.1186/s42162-021-00177-1

Cited by 6 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For this purpose, we applied the results from a cluster analysis, introduced in [42]. In [42], we introduced a "practical approach to cluster validation in the energy sector", applied the method to German municipalities, and performed multiple clusterings on the input features of the P2P-pricing dataset. For this purpose, we used so-called "clustervalidation-indices" to obtain the clustering that best fulfills the predefined characteristics.…”

Section: Model Validation Methodsmentioning

confidence: 99%

“…We use these resulting clusters to develop a representative benchmarking dataset to test the ML model results. Similar to [28], we applied a stratified random sampling using the clusters from [42] as strata. A sample of roughly 1%, or 123 of all municipalities, was taken proportionally to the size of the clusters.…”

Section: Model Validation Methodsmentioning

confidence: 99%

“…They divide their population in strata and apply an SRS on each stratum, proportional to the sample size. A drawback of this method is the more time-consuming and complex process of cluster validation, as already shown in [42].…”

Section: Stratified Samplingmentioning

confidence: 99%

“…Instead of a purely index-based internal or relative cluster validation (see [42]), we utilize an energy-economic validation approach. The goal of the time series aggregation is the representation of a year by typical periods for any municipality j individually.…”

Section: Model Evaluationmentioning

confidence: 99%

“…To avoid the clustering process for ~12,000 municipalities, we identified representative municipalities in [42]. For a growing number of typical hours, we derived the feature values (sum of the year) from the typical hours and compared them to the actual values using the APE.…”

Section: Cluster Validationmentioning

confidence: 99%

See 4 more Smart Citations

Accelerating Energy-Economic Simulation Models via Machine Learning-Based Emulation and Time Series Aggregation

2022

Self Cite

View full text Add to dashboard Cite

Energy-economic simulation models with high levels of detail, high time resolutions, or large populations (e.g., distribution networks, households, electric vehicles, energy communities) are often limited due to their computational complexity. This paper introduces a novel methodology, combining cluster-based time series aggregation and sampling methods, to efficiently emulate simulation models using machine learning and significantly reduce both simulation and training time. Machine learning-based emulation models require sufficient and high-quality data to generalize the dataset. Since simulations are computationally complex, their maximum number is limited. Sampling methods come into play when selecting the best parameters for a limited number of simulations ex ante. This paper introduces and compares multiple sampling methods on three energy-economic datasets and shows their advantage over a simple random sampling for small sample-sizes. The results show that a k-means cluster sampling approach (based on unsupervised learning) and adaptive sampling (based on supervised learning) achieve the best results especially for small sample sizes. While a k-means cluster sampling is simple to implement, it is challenging to increase the sample sizes if the emulation model does not achieve sufficient accuracy. The iterative adaptive sampling is more complex during implementation, but can be re-applied until a certain accuracy threshold is met. Emulation is then applied on a case study, emulating an energy-economic simulation framework for peer-to-peer pricing models in Germany. The evaluated pricing models are the “supply and demand ratio” (SDR) and “mid-market rate pricing” (MMR). A time series aggregation can reduce time series data of municipalities by 99.4% with less than 5% error for 98.2% (load) and 95.5% (generation) of all municipalities and hence decrease the simulation time needed to create sufficient training data. This paper combines time series aggregation and emulation in a novel approach and shows significant acceleration by up to 88.9% of the model’s initial runtime for the simulation of the entire population of around 12,000 municipalities. The time for re-calculating the population (e.g., for different scenarios or sensitivity analysis) can be increased by a factor of 1100 while still retaining high accuracy. The analysis of the simulation time shows that time series aggregation and emulation, considered individually, only bring minor improvements in the runtime but can, however, be combined effectively. This can significantly speed up both the simulation itself and the training of the emulation model and allows for flexible use, depending on the capabilities of the models and the practitioners. The results of the peer-to-peer pricing for approximately 12,000 German municipalities show great potential for energy communities. The mechanisms offer good incentives for the addition of necessary flexibility.

show abstract

Section: Model Validation Methodsmentioning

confidence: 99%

Section: Model Validation Methodsmentioning

confidence: 99%

Section: Stratified Samplingmentioning

confidence: 99%

Section: Model Evaluationmentioning

confidence: 99%

Section: Cluster Validationmentioning

confidence: 99%

See 3 more Smart Citations

Accelerating Energy-Economic Simulation Models via Machine Learning-Based Emulation and Time Series Aggregation

2022

Self Cite

View full text Add to dashboard Cite

show abstract

Application of MADM methods in Industry 4.0: A literature review

Zayat

Kılıç

Yalçın

et al. 2023

Computers & Industrial Engineering

View full text Add to dashboard Cite

Design choices in peer‐to‐peer energy markets with active network management

et al. 2022

Self Cite

View full text Add to dashboard Cite

Due to the growing number of Distributed Energy Resources and new electrical loads at the sectoral contact points, novel organisational forms such as Local Energy Markets arise to deal with increasing complexity in the energy system. However, these markets are radically different from traditional energy markets, as they often allow individual prosumers to trade with each other via a peer-to-peer scheme. To guarantee tamperproof settlement, an increasing number of these markets feature a distributed ledger technology. This paper analyses different design variants of peer-to-peer markets, focusing specifically on the allocation mechanism under network constraints as these mechanisms constitute the core component of a market design. We assess these designs concerning user acceptance, economic performance, practicability, and their ability to relieve grid congestion. Further key performance indicators also cover communal revenues or welfare distribution. For this purpose, we developed an agent-based simulation framework, which builds on data from three German reference municipalities derived from a novel clustering approach. Besides a consolidated presentation of the results, we highlight current implementation obstacles and identify promising concepts for further research.

show abstract

A practical approach to cluster validation in the energy sector

Cited by 6 publications

References 29 publications

Accelerating Energy-Economic Simulation Models via Machine Learning-Based Emulation and Time Series Aggregation

Accelerating Energy-Economic Simulation Models via Machine Learning-Based Emulation and Time Series Aggregation

Application of MADM methods in Industry 4.0: A literature review

Design choices in peer‐to‐peer energy markets with active network management

Contact Info

Product

Resources

About