Hyperparameter Optimization of LSTM Network Models through Genetic Algorithm

Gorgolis, Nikolaos; Hatzilygeroudis, Ioannis; Istenes, Zoltán; Gyenne, Lazlo n Grad

doi:10.1109/iisa.2019.8900675

Cited by 43 publications

(15 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…HPO can be seen as the final step in model design and the first step in training the neural network. Considering the effect of hyperparameters on accuracy and speed during training, the training process should be carefully experienced before starting [23]. The HPO process automatically optimizes the hyperparameters of the machine learning model to get humans out of the loop of the machine learning system.…”

Section: Hyperparameter Optimizationmentioning

confidence: 99%

An intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm for predicting concrete block production

Al-Khazraji

Nasser

Khlil

2022

IJ-AI

View full text Add to dashboard Cite

Demand forecasting aims to optimize the production planning of industrial companies by ensuring that the production planning meets the future demand. Demand forecasting utilizes historical data as an input to predict future trends of the demand. In this paper, a new approach for developing an intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm is presented. Fireﬂy algorithmbased gated recurrent units (FA-GRU) is used to tackle the production forecasting problem. The proposed model has been evaluated and compared with the standard gated recurrent unit (GRU) and standard long short-term memory model (LSTM) using historical data of 36 months of concrete block manufacturing at dler company in Iraq. The prediction accuracy of the three models is evaluated using the root mean square error (RMSE), the mean absolute percentage error (MAPE) and the statistical coefficient of determination (R2 ) indicators. The outcomes of the study show that the proposed FA-GRU gives better forecasting results compared to the standard GRU and standard LSTM.

show abstract

Section: Hyperparameter Optimizationmentioning

confidence: 99%

An intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm for predicting concrete block production

Al-Khazraji

Nasser

Khlil

2022

IJ-AI

View full text Add to dashboard Cite

show abstract

“…LSTMs have several parameters, such as the number of layers, the number of units in the hidden layer, time window size, batch size, etc., referred to as hyperparameters [18], which influence network behaviour [4] and thus should be optimised before the training process [18].…”

Section: Long-short Term Memory (Lstm) Networkmentioning

confidence: 99%

“…where n is the number of samples, y  is the desired output and ŷ is the predicted output value of the observation made by the model  th . The MAE (16), MAPE (17) and RMSE (18) were evaluated in accordance to the following equations [10]:…”

Section: F Performance Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Long Short Term Memory Water Quality Predictive Model Discrepancy Mitigation Through Genetic Algorithm Optimisation and Ensemble Modeling

2022

View full text Add to dashboard Cite

A specific long short-term memory (LSTM) model developed for water quality prediction based on a particular water quality dataset will only apply to that dataset and may fail to make an accurate prediction on another dataset. This paper focuses on improving the tolerance of LSTM prediction models by mitigating the discrepancies in model prediction capability that arise when a model is applied to different datasets. Two predictive LSTM models are developed from two different water quality datasets and are optimised using the metaheuristic genetic algorithm (GA) to create two-hybrid GA-optimised LSTM models subsequently combined using a linear weight-based technique to develop a tolerant predictive ensemble model. The hybrid models contribute equally to the average ensemble model, while one of the hybrid models has a 10% greater weight contribution in the weighted ensemble model. The ensemble models outperform the individual hybrid models, but only marginally at times. The models can successfully predict the quality of river water in terms of dissolved oxygen concentration. When tested on unseen and unrelated datasets, the models make accurate predictions and thus are applicable in domains other than the water sector. The consistent and similar performance of the models on any dataset illustrates the successful mitigation of discrepancies in the predictive capacity of individual LSTM models by the proposed ensemble scheme. Observed model performance outlined the datasets on which the models could potentially make accurate predictions.INDEX TERMS Ensemble model, environment, genetic algorithm, long short term memory, rivers, water, water quality, water conservation, weight based model fusion.

show abstract

“…One possibility is to introduce optimization techniques like grid search (GS) , random search (RS) , genetic algorithm (GA) , simulated annealing (SA) , etc. to accelerate the training process [13][14][15][16]. The above mentioned methods can improve the modelling efficiency to some extent, however, the results obtained may fluctuate due to different kinds of initialization methods.…”

Section: Introductionmentioning

confidence: 99%

Financial High-Frequency Time Series Forecasting Based on Sub-Step Grid Search Long Short-Term Memory Network

Tian

2020

IEEE Access

View full text Add to dashboard Cite

Hyperparameter Optimization of LSTM Network Models through Genetic Algorithm

Cited by 43 publications

References 4 publications

An intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm for predicting concrete block production

An intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm for predicting concrete block production

Long Short Term Memory Water Quality Predictive Model Discrepancy Mitigation Through Genetic Algorithm Optimisation and Ensemble Modeling

Financial High-Frequency Time Series Forecasting Based on Sub-Step Grid Search Long Short-Term Memory Network

Contact Info

Product

Resources

About