Wind Speed Forecasting toward El Nino Factors Using Recurrent Neural Networks

Effects of Learning Rates and Optimization Algorithms on Forecasting Accuracy of Hourly Typhoon Rainfall: Experiments With Convolutional Neural Network

“…On the other hand, the learning rate is a tuning parameter of optimization algorithms, and it controls how much to change the DL model in response to the estimated error each time the model weights are updated (Brownlee, 2020). The effects of optimization algorithms and learning rates on the forecasting performance of DL models were confirmed by previous studies (e.g., Bengio, 2012;Goodfellow et al, 2016;Prasetya & Djamal, 2019;Saputri et al, 2020). These studies reported that optimization algorithms and learning rates determined the accuracy of the DL model significantly.…”

“…More importantly, optimization algorithms such as stochastic gradient descent (SGD), root mean square propagation (RMSprop), adaptive grad (AdaGrad), adaptive delta (AdaDelta), adaptive moment estimation (Adam), adaptive maximum (Adamax), and Nesterov‐accelerated adaptive moment estimation (Nadam) are absent in the typhoon rainfall forecasting model based on the DL model (Huang et al., 2018; Lin & Chen, 2005; Lin & Wu, 2009; Wei & Chou, 2020). These optimization algorithms have successfully been applied in rainfall forecasting (Barrera‐Animas et al., 2021; Fadilah et al., 2021; Manoj & Ananth, 2020; Prasetya & Djamal, 2019; Sari et al., 2020; Zhang et al., 2018), spatial prediction of landslides (Bui et al., 2019), wind speed and wind direction forecasting (Puspita Sari et al., 2020; Saputri et al., 2020), evapotranspiration forecasting (Walls et al., 2020), run‐off forecasting (Nath et al., 2021), air quality index prediction (H. He & Luo, 2020), river stage, flash flood susceptibility and streamflow forecasting (Hitokoto et al., 2017; Rahimzad et al., 2021; Tien Bui et al., 2020), water demand forecasting (Koo et al., 2021), temperature and global solar radiation prediction (Del & Starchenko, 2021; Ghimire et al., 2019).…”

An Improved Cluster‐Wise Typhoon Rainfall Forecasting Model Based on Machine Learning and Deep Learning Models Over the Northwestern Pacific Ocean

“…Unfortunately, optimization algorithms such as stochastic gradient descent (SGD), root mean square propagation (RMSprop), adaptive grad (AdaGrad), adaptive delta (AdaDelta), adaptive moment estimation (Adam), adaptive maximum (Adamax), and Nesterov‐accelerated adaptive moment estimation (Nadam) are absent in the typhoon rainfall forecasting model based on the DL model (Lin & Chen, 2005; Lin & Wu, 2009; Wei & Chou, 2020). Both optimization algorithms and learning rates have effects on the forecasting performance of DL models, and these parameters determine the accuracy of the DL model significantly, as confirmed by studies (e.g., Bengio, 2012; Bengio et al., 2015; B. F. Chen et al., 2019; Prasetya & Djamal, 2019; Saputri et al., 2020). While too fast or an unstable training process would result in a large value of learning rate, a long training process would result in a small value of learning rate (Brownlee, 2018, 2020a, 2020b; Goodfellow et al., 2016).…”

“…Both optimization algorithms and learning rates have effects on the forecasting performance of DL models, and these parameters determine the accuracy of the DL model significantly, as confirmed by studies (e.g., Bengio, 2012;Bengio et al, 2015;B. F. Chen et al, 2019;Prasetya & Djamal, 2019;Saputri et al, 2020). While too fast or an unstable training process would result in a large value of learning rate, a long training process would result in a small value of learning rate (Brownlee, 2018(Brownlee, , 2020a(Brownlee, , 2020bGoodfellow et al, 2016).…”