A novel attention-based LSTM cell post-processor coupled with bayesian optimization for streamflow prediction

Alizadeh, Babak; Bafti, Alireza Ghaderi; Kamangir, Hamid; Zhang, Yu; Wright, Daniel B.; Franz, Kristie J.

doi:10.1016/j.jhydrol.2021.126526

Cited by 108 publications

(29 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this work, BO is used to optimize the hyperparameter configuration of the DQL-based EMS. Another interesting work is Alizadeh et al 42 , in which a novel attention-based LSTM cell has been proposed and optimized by Bayesian optimization for streamflow postprocessing which outperformed the simple LSTM, GRU, a machine learning algorithm, and two statistical-based models.…”

Section: Related Workmentioning

confidence: 99%

Bayesian optimization and deep learning for steering wheel angle prediction

Riboni

Ghioldi

Candelieri

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Automated driving systems (ADS) have undergone a significant improvement in the last years. ADS and more precisely self-driving cars technologies will change the way we perceive and know the world of transportation systems in terms of user experience, mode choices and business models. The emerging field of Deep Learning (DL) has been successfully applied for the development of innovative ADS solutions. However, the attempt to single out the best deep neural network architecture and tuning its hyperparameters are all expensive processes, both in terms of time and computational resources. In this work, Bayesian optimization (BO) is used to optimize the hyperparameters of a Spatiotemporal-Long Short Term Memory (ST-LSTM) network with the aim to obtain an accurate model for the prediction of the steering angle in a ADS. BO was able to identify, within a limited number of trials, a model—namely BO_ST-LSTM—which resulted, on a public dataset, the most accurate when compared to classical end-to-end driving models.

show abstract

Section: Related Workmentioning

confidence: 99%

Bayesian optimization and deep learning for steering wheel angle prediction

Riboni

Ghioldi

Candelieri

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Ref. [26] implements the BO algorithm for the attention-based LSTM model for stream-flow prediction. Ref.…”

Section: Introductionmentioning

confidence: 99%

Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting

Zulfiqar

Gamage

Kamran

et al. 2022

Sensors

View full text Add to dashboard Cite

This paper proposes a new hybrid framework for short-term load forecasting (STLF) by combining the Feature Engineering (FE) and Bayesian Optimization (BO) algorithms with a Bayesian Neural Network (BNN). The FE module comprises feature selection and extraction phases. Firstly, by merging the Random Forest (RaF) and Relief-F (ReF) algorithms, we developed a hybrid feature selector based on grey correlation analysis (GCA) to eliminate feature redundancy. Secondly, a radial basis Kernel function and principal component analysis (KPCA) are integrated into the feature-extraction module for dimensional reduction. Thirdly, the Bayesian Optimization (BO) algorithm is used to fine-tune the control parameters of a BNN and provides more accurate results by avoiding the optimal local trapping. The proposed FE-BNN-BO framework works in such a way to ensure stability, convergence, and accuracy. The proposed FE-BNN-BO model is tested on the hourly load data obtained from the PJM, USA, electricity market. In addition, the simulation results are also compared with other benchmark models such as Bi-Level, long short-term memory (LSTM), an accurate and fast convergence-based ANN (ANN-AFC), and a mutual-information-based ANN (ANN-MI). The results show that the proposed model has significantly improved the accuracy with a fast convergence rate and reduced the mean absolute percent error (MAPE).

show abstract

“…Exploring optimal hyperparameters for an LSTM model is already a study objective in applications of LSTM models in fields other than hydrology, such as sequence labeling [18], network attack detection [19], stock market prediction [20], highway traffic prediction [21], etc. In the case of hydrology, the tuning of hyperparameters is a necessary step before the application of the model in other research [4,5,[22][23][24][25]; however, no sufficient knowledge has been obtained.…”

Section: Introductionmentioning

confidence: 99%

Investigation of Hyperparameter Setting of a Long Short-Term Memory Model Applied for Imputation of Missing Discharge Data of the Daihachiga River

Weilisi

Kojima

2022

Water

View full text Add to dashboard Cite

Missing observational data pose an unavoidable problem in the hydrological field. Deep learning technology has recently been developing rapidly, and has started to be applied in the hydrological field. Being one of the network architectures used in deep learning, Long Short-Term Memory (LSTM) has been applied largely in related research, such as flood forecasting and discharge prediction, and the performance of an LSTM model has been compared with other deep learning models. Although the tuning of hyperparameters, which influences the performance of an LSTM model, is necessary, no sufficient knowledge has been obtained. In this study, we tuned the hyperparameters of an LSTM model to investigate the influence on the model performance, and tried to obtain a more suitable hyperparameter combination for the imputation of missing discharge data of the Daihachiga River. A traditional method, linear regression with an accuracy of 0.903 in Nash–Sutcliffe Efficiency (NSE), was chosen as the comparison target of the accuracy. The results of most of the trainings that used the discharge data of both neighboring and estimation points had better accuracy than the regression. Imputation of 7 days of the missing period had a minimum value of 0.904 in NSE, and 1 day of the missing period had a lower quartile of 0.922 in NSE. Dropout value indicated a negative correlation with the accuracy. Setting dropout as 0 had the best accuracy, 0.917 in the lower quartile of NSE. When the missing period was 1 day and the number of hidden layers were more than 100, all the compared results had an accuracy of 0.907–0.959 in NSE. Consequently, the case, which used discharge data with backtracked time considering the missing period of 1 day and 7 days and discharge data of adjacent points as input data, indicated better accuracy than other input data combinations. Moreover, the following information is obtained for this LSTM model: 100 hidden layers are better, and dropout and recurrent dropout levels equaling 0 are also better. The obtained optimal combination of hyperparameters exceeded the accuracy of the traditional method of regression analysis.

show abstract

A novel attention-based LSTM cell post-processor coupled with bayesian optimization for streamflow prediction

Cited by 108 publications

References 53 publications

Bayesian optimization and deep learning for steering wheel angle prediction

Bayesian optimization and deep learning for steering wheel angle prediction

Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting

Investigation of Hyperparameter Setting of a Long Short-Term Memory Model Applied for Imputation of Missing Discharge Data of the Daihachiga River

Contact Info

Product

Resources

About