Feature Selection and Extraction Along with Electricity Price Forecasting Using Big Data Analytics

Shafi, Isra; Javaid, Nadeem; Naz, Aqdas; Amir, Yasir; Ishaq, Israr; Naseem, Kashif

doi:10.1007/978-3-319-93554-6_27

Cited by 3 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides the variables selection and contribution rate, this study also trained data with only the variables considered most important. This could have been applied in the work Bannor and Acheampong (2019), as the selection of variables can identify a subset of the most relevant information in the system, and therefore influence the energy prediction accuracy (Yang et al, 2019;Peres and Fogliatto, 2018;Abedinia et al, 2016;Shafi et al, 2019;Li et al, 2019). The approach used in this work not only eliminates the variables that hinder the model, but also calculates the contribution rate for each variable.…”

Section: Resultsmentioning

confidence: 99%

“…Initially, the prediction was performed with 26 input variables, leaving the energy load variable GWh as output and, subsequently, according to the contribution rate of variables, several prediction tests were made. However, as stated earlier in Section 2, it is essential to select optimal input capabilities by removing irrelevant data to facilitate future analysis and to improve the accuracy of energy load forecasting (Shafi et al , 2019; Abedinia et al , 2016; Yang et al , 2019).…”

Section: Methodsmentioning

confidence: 99%

“…Variable selection methods are fundamental because they can identify a subset of explanatory variables that contain the most relevant information in the complete data set and, therefore, influence the accuracy of the forecast (Yang et al , 2019; Peres and Fogliatto, 2018; Abedinia et al , 2016; Shafi et al , 2019; Li et al , 2019). Aware of this importance, some authors have explored the use of attribute selection techniques mainly to accelerate training and increase accuracy of forecasting models (Hafeez et al , 2021; Kim et al , 2020; Dai and Zhao, 2020).…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A PCA-based variable ranking and selection approach for electric energy load forecasting

Bezerra

Grassi

Dias

et al. 2022

IJESM

View full text Add to dashboard Cite

Purpose This paper aims to propose an approach based upon the principal component analysis (PCA) to define a contribution rate for each variable and then select the main variables as inputs to a neural network for energy load forecasting in the region southeastern Brazil. Design/methodology/approach The proposed approach defines a contribution rate of each variable as a weighted sum of the inner product between the variable and each principal component. So, the contribution rate is used for selecting the most important features of 27 variables and 6,815 electricity data for a multilayer perceptron network backpropagation prediction model. Several tests, starting from the most significant variable as input, and adding the next most significant variable and so on, are accomplished to predict energy load (GWh). The Kaiser–Meyer–Olkin and Bartlett sphericity tests were used to verify the overall consistency of the data for factor analysis. Findings Although energy load forecasting is an area for which databases with tens or hundreds of variables are available, the approach could select only six variables that contribute more than 85% for the model. While the contribution rates of the variables of the plants, plus energy exchange added, have only 14.14% of contribution, the variable the stored energy has a contribution rate of 26.31% being fundamental for the prediction accuracy. Originality/value Besides improving the forecasting accuracy and providing a faster predictor, the proposed PCA-based approach for calculating the contribution rate of input variables providing a better understanding of the underlying process that generated the data, which is fundamental to the Brazilian reality due to the accentuated climatic and economic variations.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A PCA-based variable ranking and selection approach for electric energy load forecasting

Bezerra

Grassi

Dias

et al. 2022

IJESM

View full text Add to dashboard Cite

show abstract

“…It also used to estimate the importance of features. We apply the ExtraTreeClassifier class (i.e., scikit-learn python library) to select features from the Pima Indian Diabetes dataset [25,26].…”

Section: Feature Importance Selection Methods Is a Technique Derived Fmentioning

confidence: 99%

Predicting Diabetes from Health-based Streaming Data using Social Media, Machine Learning and Stream Processing Technologies

Hassan¹,

Shaheen²

2020

International Journal of Engineering Research and Technology

View full text Add to dashboard Cite

Diabetes disease is one of the main healthcare challenges in all the world. Undiagnosed diabetes can increase the danger of cardiac stroke, diabetic nephropathy, and other disorders. Early detection of diabetes is necessary to take care of a healthy life. Nowadays, social media is a new dimension to deal with health care by exploiting the real-time shared patients' data to early detect diabetes disease. Furthermore, technologies typically associated with digitalization add value in healthcare, including artificial intelligence, data analytic technologies, and stream processing technologies. Therefore, in this research, we propose a real-time system for predicting diabetes disease from health-based social streaming data to indicate the current status for patient health. The proposed system aims to find the most accurate machine learning model which has the highest accuracy of diabetes prediction. We have used three types of feature selection techniques to select the most relevant features from the used dataset i.e., Recursive Feature Elimination, Univariate feature selection, and Feature Importance. Also, we have evaluated and compared four machine learning models with selected and full features i.e, , Random Forest, Support Vector Machine, Decision Tree, and Logistic Regression Classifier. The experimental results have determined that the random forest model has achieved the greatest accuracy among other models at 84.11%. For online prediction through social media, we have performed our proposed system to handle streaming Twitter data about patients' health. In doing so, Kafka and Spark streaming are integrated into the backend of the proposed system. Then, the random forest classifier is used to predict the patient's current health status in real-time.

show abstract

“…State estimation Principal Component Analysis (PCA) algorithm [23] State estimation t-distributed stochastic neighbor embedding method [24] Feature extraction Pearson Correlation Coeflcient based Extra tree classifier [24] Feature selection to remove irrelevant data Support Vector Machine (SVM) classifier [24] Electricity price forecasting Sparse auto encoder [25] Over voltage identification based on feature extraction Multi level Discrete Wavelet Transform (DWT) [26] Dimensionality reduction of daily load curve Fuzzy Based Feature Selection (FBFS) [27] Feature selection Swinging Door Trending (SDT) [28] PMU data compression Principal Component Analysis (PCA) [29][30][31] Dimensionality reduction of PMU data Event oriented auto-adjustable sliding window method [32] Event detection and AMI), historical data and forecasting data (weather, consumer and generation power patterns). The sizes of these heterogeneous data are measured in terabytes and petabytes, which causes congestion and requires an increased bandwidth for the communication paths.…”

Section: Dimensionality Reduction Technique Applicationmentioning

confidence: 99%

The Paradigm Revolution in the Distribution Grid: The Cutting-Edge and Enabling Technologies

Thasnimol

Rajathy

2020

Open Computer Science

View full text Add to dashboard Cite

Bi-directional information and energy flow, renewable energy sources, battery energy storage, electric vehicle, self-healing capability, and demand response programs, etc., revolutionized the traditional distribution network into the smart distribution network. Adoption of modern technologies like intelligent meters such as advanced metering infrastructure & Micro-phasor measurement units, data storage, and analysis techniques and incentive-based electricity trading mechanisms can bring this paradigm shift. This study presents an overview of popular technologies that facilitate this transformation, giving focus on some prime technologies such as real-time monitoring based on Micro-phasor measurement units, data storage and analytics, blockchain technology, multi-agent systems, and incentive-based energy trading mechanisms

show abstract

Feature Selection and Extraction Along with Electricity Price Forecasting Using Big Data Analytics

Cited by 3 publications

References 13 publications

A PCA-based variable ranking and selection approach for electric energy load forecasting

A PCA-based variable ranking and selection approach for electric energy load forecasting

Predicting Diabetes from Health-based Streaming Data using Social Media, Machine Learning and Stream Processing Technologies

The Paradigm Revolution in the Distribution Grid: The Cutting-Edge and Enabling Technologies

Contact Info

Product

Resources

About