Artificial Neural Network Hyperparameters Optimization: A Survey

Kadhim, Zahraa Saddi; Abdullah, Hasanen S.; Ghathwan, Khalil Ibrahim

doi:10.3991/ijoe.v18i15.34399

Cited by 14 publications

(8 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One of the challenges in designing the CIP is to tune the hyperparameters for the base model in Level 0, as there is no single best set of hyperparameters due to the complex relationships between them, requiring trial and error to find the best combination [25]. CIP utilizes Bayesian optimization as it utilizes new parameter combinations and exploits known promising regions to navigate the optimization space efficiently [26], which could shorten the computation time and guarantee an optimized outcome.…”

Section: Hyperparameter Optimizationmentioning

confidence: 99%

Resilient Electricity Load Forecasting Network with Collective Intelligence Predictor for Smart Cities

Bin Kamilin,

Yamaguchi

2024

Electronics

View full text Add to dashboard Cite

Accurate electricity forecasting is essential for smart cities to maintain grid stability by allocating resources in advance, ensuring better integration with renewable energies, and lowering operation costs. However, most forecasting models that use machine learning cannot handle the missing values and possess a single point of failure. With rapid technological advancement, smart cities are becoming lucrative targets for cyberattacks to induce packet loss or take down servers offline via distributed denial-of-service attacks, disrupting the forecasting system and inducing missing values in the electricity load data. This paper proposes a collective intelligence predictor, which uses modular three-level forecasting networks to decentralize and strengthen against missing values. Compared to the existing forecasting models, it achieves a coefficient of determination score of 0.98831 with no missing values using the base model in the Level 0 network. As the missing values in the forecasted zone rise to 90% and a single-model forecasting method is no longer effective, it achieves a score of 0.89345 with a meta-model in the Level 1 network to aggregate the results from the base models in Level 0. Finally, as missing values reach 100%, it achieves a score of 0.81445 by reconstructing the forecast from other zones using the meta-model in the Level 2 network.

show abstract

Section: Hyperparameter Optimizationmentioning

confidence: 99%

Resilient Electricity Load Forecasting Network with Collective Intelligence Predictor for Smart Cities

Bin Kamilin,

Yamaguchi

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…In simple terms, NN is a non-linear statistical data modeling tool. NN can be used to model complex relationships between input and output to find patterns in data [11] [21]. BPNN is a method developed by adding hidden for the backward process in NN.…”

Section: Algoritma Momentum Backpropagationmentioning

confidence: 99%

Random Search Hyperparameter Optimization for BPNN to Forecasting Cattle Population

Khotimah,

Agustina,

Puspitarini

et al. 2024

E3S Web Conf.

View full text Add to dashboard Cite

Backpropagation Neural Network (BPNN) is a suitable method for predicting the future. It has weaknesses, namely poor convergence speed and instability, requiring parameter tuning to overcome speed problems, and having a high bias. This research uses the Random Search hyperparameter technique to optimize BPNN to automatically select the number of hidden layers, learning rate, and momentum. The added accuracy of momentum will speed up the training process, produce predictions with better accuracy, and determine the best architectural model from a series of faster training processes with low bias. This research will predict the local Indonesian cattle population, which is widely developed by people in the eastern part, especially Madura, in 4 types of cattle: sono cattle, karapan cattle, mixed cattle, and breeder cattle. The results of BPNN hyperparameter measurements with the best model show that hyperparameter optimization did not experience overfitting and experienced an increase in accuracy of 2.5% compared to the Neural Network model without hyperparameter optimization. Based on the test results, the BPNN algorithm parameters with a data ratio of 70:30, the best architecture for backpropagation momentum is 6-6-1, with a learning rate of 0.002, momentum 0.3, which has an MSE during testing of 0.1176 on Karapan type Madurese cattle. Tests based on computing time measurements show that the BPNN hyperparameter algorithm stops at 490 iterations compared to regular BPNN. The research results show that the hidden layers, learning rate, and momentum if optimized simultaneously, have a significant influence in preventing overfitting, increasing accuracy, and having better execution times than without optimization.

show abstract

“…Ref. [61] analyzes the characteristics of different types of artificial neural networks, while [62] investigates research on spiking neural networks that are more inclusive of time series. Finally, after determining the plans for all the above aspects, the integration framework needs to select an appropriate training strategy for reinforcement learning according to task requirements.…”

Section: Integrated Characteristic Analysismentioning

confidence: 99%

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Wu,

Zhang,

Miao

et al. 2024

Electronics

View full text Add to dashboard Cite

Extensive research has been carried out on reinforcement learning methods. The core idea of reinforcement learning is to learn methods by means of trial and error, and it has been successfully applied to robotics, autonomous driving, gaming, healthcare, resource management, and other fields. However, when building reinforcement learning solutions at the edge, not only are there the challenges of data-hungry and insufficient computational resources but also there is the difficulty of a single reinforcement learning method to meet the requirements of the model in terms of efficiency, generalization, robustness, and so on. These solutions rely on expert knowledge for the design of edge-side integrated reinforcement learning methods, and they lack high-level system architecture design to support their wider generalization and application. Therefore, in this paper, instead of surveying reinforcement learning systems, we survey the most commonly used options for each part of the architecture from the point of view of integrated application. We present the characteristics of traditional reinforcement learning in several aspects and design a corresponding integration framework based on them. In this process, we show a complete primer on the design of reinforcement learning architectures while also demonstrating the flexibility of the various parts of the architecture to be adapted to the characteristics of different edge tasks. Overall, reinforcement learning has become an important tool in intelligent decision making, but it still faces many challenges in the practical application in edge computing. The aim of this paper is to provide researchers and practitioners with a new, integrated perspective to better understand and apply reinforcement learning in edge decision-making tasks.

show abstract

Artificial Neural Network Hyperparameters Optimization: A Survey

Cited by 14 publications

References 72 publications

Resilient Electricity Load Forecasting Network with Collective Intelligence Predictor for Smart Cities

Resilient Electricity Load Forecasting Network with Collective Intelligence Predictor for Smart Cities

Random Search Hyperparameter Optimization for BPNN to Forecasting Cattle Population

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Contact Info

Product

Resources

About