A Survey of Preprocessing Methods Used for Analysis of Big Data Originated From Smart Grids

Alghamdi, Turki Ali; Javaid, Nadeem

doi:10.1109/access.2022.3157941

Cited by 24 publications

(16 citation statements)

References 108 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it does not demand an extensive depth of expertise in either theoretical methods or domain knowledge, making it more versatile. This method does not involve the application of complex forecasting algorithms [3][4][5][6] or require a strong theoretical development background [7][8][9][10][11]. Its core lies in the calculation of the error degree "E" in Equation ( 1), the consideration of energy consumption-environment related factors (Figure 3), and the correlation calculation step (Figure 4).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Zero-Carbon Building Operation and Maintenance: A Correlation-Based Data Mining Approach for Database Analysis

Zhao,

Liu,

Liu

et al. 2023

Sustainability

View full text Add to dashboard Cite

In the context of global climate change and the increasing focus on carbon emissions, carbon emission research has become a prominent area of study. However, research in this field inevitably involves extensive monitoring, and when the data become complex and chaotic, the accuracy of these data can be challenging to control, making it difficult to determine their reliability. This article starts by exploring the operational and maintenance data of zero-carbon buildings, aiming to uncover the correlation between energy consumption data and environmental data. This correlation is categorized into two main types: linear correlation and trend correlation. By establishing error degree calculations based on these correlation relationships, anomaly detection can be performed on the data. Analyzing the interrelationships between these datasets allows for the formulation of appropriate fitting equations, primarily consisting of linear and polynomial fits, all of which exhibit a determination coefficient exceeding 0.99. These fitting equations are then utilized to correct errors in the anomalous data, and the reasonableness of the fitting methods is demonstrated by examining the residual distribution. The final results align with the corresponding expectations, providing a concise and effective correction method for monitoring data in zero-carbon smart buildings. Importantly, this method exhibits a certain level of generality and can be applied to various scenarios within the realm of zero-carbon buildings.

show abstract

Section: Resultsmentioning

confidence: 99%

“…The proposed method is more accurate for low-energy and net-zero-energy buildings. Alghamdi and Javaid [6] significantly enhanced the accuracy of subsequent predictions by preprocessing data related to smart grid loads and prices.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Zero-Carbon Building Operation and Maintenance: A Correlation-Based Data Mining Approach for Database Analysis

Zhao,

Liu,

Liu

et al. 2023

Sustainability

View full text Add to dashboard Cite

show abstract

“…Paramount quantities of IoT data require powerful AI techniques for pre-processing and preparing data to reduce noise, minimize dimensionality, and remove possible redundancies [ 26 ]. In most reports, AI techniques such as artificial neural networks, fuzzy logic, and evolutionary computation are mainly used for heterogeneous purposes [ 27 ], including classification, regression, signal processing, forecasting, decision support, and data transmission.…”

Section: Introductionmentioning

confidence: 99%

Recent Advancements and Challenges of AIoT Application in Smart Agriculture: A Review

Adli

Remli

Wong

et al. 2023

Sensors

View full text Add to dashboard Cite

As the most popular technologies of the 21st century, artificial intelligence (AI) and the internet of things (IoT) are the most effective paradigms that have played a vital role in transforming the agricultural industry during the pandemic. The convergence of AI and IoT has sparked a recent wave of interest in artificial intelligence of things (AIoT). An IoT system provides data flow to AI techniques for data integration and interpretation as well as for the performance of automatic image analysis and data prediction. The adoption of AIoT technology significantly transforms the traditional agriculture scenario by addressing numerous challenges, including pest management and post-harvest management issues. Although AIoT is an essential driving force for smart agriculture, there are still some barriers that must be overcome. In this paper, a systematic literature review of AIoT is presented to highlight the current progress, its applications, and its advantages. The AIoT concept, from smart devices in IoT systems to the adoption of AI techniques, is discussed. The increasing trend in article publication regarding to AIoT topics is presented based on a database search process. Lastly, the challenges to the adoption of AIoT technology in modern agriculture are also discussed.

show abstract

“…Despite its significant importance, data preprocessing reduces the portability of ML models and incurs extra computational overhead for ML tasks, limiting their applicability to resource-constrained platforms [5,6]. This work presents a methodology to implement several signal processing-enabled preprocessing tasks inside DNN models using CLs rather than separate preceding pipelines.…”

Section: Introductionmentioning

confidence: 99%

Design and Analysis of Convolutional Neural Layers: A Signal Processing Perspective

Farag

2023

IEEE Access

View full text Add to dashboard Cite

Convolutional layers (CLs) are ubiquitous in contemporary deep neural network (DNN) models, commonly used for automatic feature extraction. A CL performs cross-correlation between the input to the layer and a set of learnable kernels to produce the layer output. Typically, kernel weights are randomly initialized and automatically learned during model training using the backpropagation and gradient descent algorithms to minimize a specific loss function. Modern DNN models comprise deep hierarchical stacks of CLs and pooling layers. Despite their prevalence, CLs are perceived as a magical tool for feature extraction without solid interpretations of their underlying working principles. In this work, we advance a method for designing and analyzing CLs by providing novel signal processing interpretations of the CL by exploiting the correlation and equivalent convolution functions of the layer. The proposed interpretations enable the employment of CLs to develop finite impulse response (FIR) filters, matched filters (MFs), short-time Fourier transform (STFT), discrete-time Fourier transform (DTFT), and continuous wavelet transform (CWT) algorithms. The main idea is to pre-assign the CL kernel weights to implement a specific convolution-or correlation-based DSP algorithm. Such an approach enables building self-contained DNN models in which CLs are utilized for various preprocessing and feature extractions tasks, enhancing the model portability, and cutting down the preprocessing computational cost. The proposed DSP interpretations provide an effective means to analyze and explain the operation of automatically trained CLs in the time and frequency domains by reversing the design procedures. The presented interpretations are mathematically established and experimentally validated with a comprehensive machinery fault diagnosis application example illustrating the potential of the proposed methodology.

show abstract

A Survey of Preprocessing Methods Used for Analysis of Big Data Originated From Smart Grids

Cited by 24 publications

References 108 publications

Enhancing Zero-Carbon Building Operation and Maintenance: A Correlation-Based Data Mining Approach for Database Analysis

Enhancing Zero-Carbon Building Operation and Maintenance: A Correlation-Based Data Mining Approach for Database Analysis

Recent Advancements and Challenges of AIoT Application in Smart Agriculture: A Review

Design and Analysis of Convolutional Neural Layers: A Signal Processing Perspective

Contact Info

Product

Resources

About