Forecasting Smart Meter Energy Usage Using Distributed Systems and Machine Learning

Dong, Chris; Du, Lingzhi; Ji, Feiran; Song, Zizhen; Zheng, Yuedi; Intrevado, Paul; Woodbridge, Diane; Howard, Alexander J.

doi:10.1109/hpcc/smartcity/dss.2018.00216

Cited by 16 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the experiments, the authors rely on simulated and real data, e.g., from publicly available datasets such as the Reference Energy Disaggregation dataset. Dong et al [41] present an approach for predicting single-day energy consumption based on a Random Forest model. This work is implemented using a publicly available dataset from the Low Carbon London project.…”

Section: E Power Grid Operationmentioning

confidence: 99%

A Framework for Enabling Cloud Services to Leverage Energy Data

Karagiannis,

Kashyap,

Zechner

et al. 2023

2023 IEEE International Conference on Cloud Engineering (IC2E)

View full text Add to dashboard Cite

Cloud services have been well integrated into various sectors of contemporary societies such as transportation, healthcare, and work. However, the energy sector has made little progress in offering cloud services to energy consumers. In some cases, even basic features (such as energy consumption monitoring and visualization of households) are still unattainable. To foster innovation in cloud-based energy services, governments implement regulations that moderate access to energy data via the energy providers' platforms. Despite these regulations, cloud service providers may still struggle to aggregate energy data because each energy provider may be using different procedures, authentication mechanisms, and data semantics. Consequently, aggregating energy data at scale becomes challenging, as it requires achieving compatibility with diverse platforms. To address this challenge, we propose a cloud-based framework that handles all interactions with energy providers and prepares the data for further processing by services. Additionally, we analyze the socioeconomic impact of our approach, and we outline novel use cases that may emerge due to the proposed framework.

show abstract

Section: E Power Grid Operationmentioning

confidence: 99%

A Framework for Enabling Cloud Services to Leverage Energy Data

Karagiannis,

Kashyap,

Zechner

et al. 2023

2023 IEEE International Conference on Cloud Engineering (IC2E)

View full text Add to dashboard Cite

show abstract

“…DML algorithms (e.g., variants of Decentralized Kalman Filters, Decentralized Alternating Least Square Recommenders, and Decentralized Mini-Batch SGD-based Classifiers), and calculation and ML platforms with strong scalability (e.g., Apache Spark, Spark MLlib, Spark Streaming, and Python/PySpark) are integrated to build a high-performance, decentralized, and fault-tolerant architecture. Dong et al [131] develop a forecasting model to predict future electricity cost of residents via a Random Forest ML algorithm. Distributed systems, such as Amazon Web Service (AWS), Simple Storage Service (S3), Elastic Map Reduce (EMR), MongoDB and Apache Spark, are used to store and process residential energy usage data collected by a smart meter in London.…”

Section: A Mapreducementioning

confidence: 99%

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications. This is due to improved capabilities of terminal devices, explosively growing data volume, congestion in the radio interfaces, and increasing concern of data privacy. The unique features of wireless systems, such as large scale, geographically dispersed deployment, user mobility, and massive amount of data, give rise to new challenges in the design of DML techniques. There is a clear gap in the existing literature in that the DML techniques are yet to be systematically reviewed for their applicability to wireless systems. This survey bridges the gap by providing a contemporary and comprehensive survey of DML techniques with a focus on wireless networks. Specifically, we review the latest applications of DML in power control, spectrum management, user association, and edge cloud computing. The optimality, scalability, convergence rate, computation cost, and communication overhead of DML are analyzed. We also discuss the potential adversarial attacks faced by DML applications, and describe state-of-the-art countermeasures to preserve privacy and security. Last but not least, we point out a number of key issues yet to be addressed, and collate potentially interesting and challenging topics for future research.

show abstract

“…A reduce function takes the outputs of the map function as input from multiple machines and executes a summary operation, returning a single answer to a driver [13]. MapReduce is a highly efficient model, and it and its variations are actively used in both research and industry [14], [15], [16].…”

Section: Introductionmentioning

confidence: 99%

Predicting Unethical Physician Behavior At Scale: A Distributed Computing Framework

Keck

Romero

Schleicher

et al. 2019

2019 IEEE SmartWorld, Ubiquitous Intelligence &Amp; Computing, Advanced &Amp; Trusted Computing, Scalable Computing &Amp; Commu

Self Cite

View full text Add to dashboard Cite

As the amount of publicly shared data increases, developing a robust pipeline to stream, store and process data is critical, as the casual user often lacks the technology, hardware and/or skills needed to work with such voluminous data. In this research, the authors employ Amazon EC2 and EMR, MongoDB, and Spark MLlib to explore 28.5 gigabytes of CMS Open Payments data in an attempt to identify physicians who may have a high propensity to act unethically, owing to significant transfers of wealth from medical companies. A Random Forest Classifier is employed to predict the top decile of physicians who have the highest risk of unethical behavior in the following year, resulting in an F-Score of 91%. The data is also analyzed by an anomaly detection algorithm that correctly identified a highprofile case of a physician leaving his prestigious position, as he failed to disclose anomalously-large transfers of wealth from medical companies.

show abstract

Forecasting Smart Meter Energy Usage Using Distributed Systems and Machine Learning

Cited by 16 publications

References 10 publications

A Framework for Enabling Cloud Services to Leverage Energy Data

A Framework for Enabling Cloud Services to Leverage Energy Data

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Predicting Unethical Physician Behavior At Scale: A Distributed Computing Framework

Contact Info

Product

Resources

About