Optimizing deep neural networks hyperparameter positions and values

Akl, Ahmed; El-Henawy, Ibrahim M.; Salah, Ahmad; Li, Kenli

doi:10.3233/jifs-190033

Cited by 14 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study varied hyperparameters/parameters like optimization algorithms, momentum, activation function, number of layers, dropout, learning rate, epoch, batch size, and others in a very systemic but proficient manner. We understand that there are different approaches used in literature for determining the optimal set of hyperparameters in deep learning model 23,50–53 . Approaches for finding out the best hyperparameters are grid search, 27 random search, 28 manual method genetic algorithm, Bayesian method, with new approaches evolving which rely on algorithms and optimization methods.…”

Section: Overview Of the Proposed Approachmentioning

confidence: 99%

A comparative performance study of random‐grid model for hyperparameters selection in detection of abnormalities in digital breast images

Oyelade

Ezugwu

2022

Concurrency and Computation

View full text Add to dashboard Cite

Deep learning models have been widely reported to have achieved significant performance in image processing and classification tasks. They have mainly been harnessed and applied to the problem of detecting abnormalities in digital breast images. However, the significant number and high dimensional space requirement of hyperparameters in deep learning models often make it challenging to find the best configuration for such parameters when tuning for best performance. In appropriately handling this performance tuning often lead to difficulty in striking balance between underfitting and overfitting. This article proposes an optimized convolutional neural network (CNN) architecture through application of hybrid selection model in obtaining best hyperparameter configuration which outperforms similar existing models. We approached this non-trivial challenge by defining a hybrid of the grid-based and random-based model for the selection of hyperparameters and then investigate the performance of the configurations. To further improve the performance of the CNN model, data augmentation technique was applied. Furthermore, the study undertook a comparative study of the performance of the best configuration on some benchmarked datasets. The resulting model was applied to publicly available benchmark datasets, namely, the DDSM and MIAS datasets. Findings from the experimentations revealed that hyperparameters with Adam optimization algorithm showed superiority by yielding an accuracy of 1.0 using DDSM dataset, while SGD, RMSprop, Adam, and Adagrad output an accuracy of 0.9375 with MIAS dataset. The outcome of this study further strengthens the appropriateness of Adam optimizer and has also produced a state-of-the-art CNN model suitable for solving the problem of detection and classification of breast cancer from digital mammography.

show abstract

Section: Overview Of the Proposed Approachmentioning

confidence: 99%

A comparative performance study of random‐grid model for hyperparameters selection in detection of abnormalities in digital breast images

Oyelade

Ezugwu

2022

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Akl et al [37], in their work, studied the effect of altering a hyperparameter within the deep learning model architecture.…”

Section: Related Workmentioning

confidence: 99%

Impact of Parameter Tuning for Optimizing Deep Neural Network Models for Predicting Software Faults

Gupta

Rajnish

Bhattacherjee

2021

Scientific Programming

View full text Add to dashboard Cite

Deep neural network models built by the appropriate design decisions are crucial to obtain the desired classifier performance. This is especially desired when predicting fault proneness of software modules. When correctly identified, this could help in reducing the testing cost by directing the efforts more towards the modules identified to be fault prone. To be able to build an efficient deep neural network model, it is important that the parameters such as number of hidden layers, number of nodes in each layer, and training details such as learning rate and regularization methods be investigated in detail. The objective of this paper is to show the importance of hyperparameter tuning in developing efficient deep neural network models for predicting fault proneness of software modules and to compare the results with other machine learning algorithms. It is shown that the proposed model outperforms the other algorithms in most cases.

show abstract

“…Hyperparameter optimization is an essential step in the implementation of any machine learning model [22]. This optimization process includes regularly modifying the model's hyperparameter values to minimize the testing error.…”

Section: Hyperparameter Tuningmentioning

confidence: 99%

“…This optimization process includes regularly modifying the model's hyperparameter values to minimize the testing error. Based on Ahmed A. Akl et al research, kernel initializer and batch size rate need to optimize for better accuracy [22]. Meanwhile, Kwon, D. H. proposed dropout rate, neuron units, and learning optimizer [23].…”

Section: Hyperparameter Tuningmentioning

confidence: 99%

Forecasting Gold Price in Rupiah using Multivariate Analysis with LSTM and GRU Neural Networks

Primananda¹,

Isa²

2021

Adv. sci. technol. eng. syst. j.

View full text Add to dashboard Cite

Forecasting the gold price movement's volatility has essential applications in areas such as risk management, options pricing, and asset allocation. The multivariate model is expected to generate more accurate forecasts than univariate models in time series data like gold prices. Multivariate analysis is based on observation and analysis of more than one statistical variable at a time. This paper mainly builds a multivariate prediction model based on Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) model to analyze and forecast the price of the gold commodity. In addition, the prediction model is optimized with a Cross-Validated Grid Search to find the optimum hyperparameter. The empirical results show that the proposed Timeseries Prediction model has an excellent accuracy in prediction, that proven by the lowest Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE). Overall, in more than three years data period, LSTM has high accuracy, but for under three years period, GRU does better. This research aims to find a promising methodology for gold price forecasting with high accuracy.

show abstract

Optimizing deep neural networks hyperparameter positions and values

Cited by 14 publications

References 14 publications

A comparative performance study of random‐grid model for hyperparameters selection in detection of abnormalities in digital breast images

A comparative performance study of random‐grid model for hyperparameters selection in detection of abnormalities in digital breast images

Impact of Parameter Tuning for Optimizing Deep Neural Network Models for Predicting Software Faults

Forecasting Gold Price in Rupiah using Multivariate Analysis with LSTM and GRU Neural Networks

Contact Info

Product

Resources

About