Using Metaheuristics for Hyper-Parameter Optimization of Convolutional Neural Networks

Bibaeva, Victoria

doi:10.1109/mlsp.2018.8516989

Cited by 25 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, recent study (Sun et al, 2018) shows that particle swarm optimization (PSO) shows higher performance than grid search. In addition, metaheuristic algorithms do not require gradients and convexity of the problem while finding global optimum (Stojanovic et al, 2016), and their usage on hyperparameter tuning has been reported recently (Bibaeva, 2018). Although practical application using autoencoder reviewed in this paper did not demonstrate hyperparameter tuning, the framework in this paper can be improved with appropriate hyperparameter tuning method.…”

Section: Resultsmentioning

confidence: 95%

Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor

et al. 2021

View full text Add to dashboard Cite

Sound and vibration analysis are prominent tools for machine health diagnosis. Especially, neural network (NN) strategies have focused on finding complex and nonlinear relationships between the sensor signal and the machine status to detect machine faults. However, it is difficult to collect enough amount of fault data as much as normal status data for training general NN models. To resolve the issue, this paper proposes the autoencoder-based anomaly detection framework for industrial robot arms using an internal sound sensor. The autoencoder uses signals in the normal state of the robots for training the model. It reconstructs the input signals as output, and anomalous states are found from high reconstruction error. Two stethoscopes were attached to the surface of the robot joint as sensors, and the sounds were recorded by USB microphone attached to the outlet of the stethoscopes. Features were extracted from STFT spectrogram images of the gathered sound, then used to train and test an autoencoder model. The reconstruction errors of the autoencoder were compared to distinguish the abnormal status from normal one. The experimental results suggest that the stethoscopes prevent the interference of noise, and the collected sound signals can be utilized for detecting machine anomalies.

show abstract

Section: Resultsmentioning

confidence: 95%

Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor

et al. 2021

View full text Add to dashboard Cite

show abstract

“…However, they did not perform infield testing to evaluate the performance of the best performing CNN model architecture obtained after the hand-tuning of architectures. Moreover, tuning CNN using a metaheuristic approach has proven its competence as compared to manually tuning it [31]. Jaddi et al [32] have used a bat algorithm to optimize the architecture, as well as the weights and biases of simple feedforward neural networks.…”

Section: Related Workmentioning

confidence: 99%

Optimizing Steering Angle Predictive Convolutional Neural Network for Autonomous Car

Saleem¹,

Riaz²,

Shaikh³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Deep learning techniques, particularly convolutional neural networks (CNNs), have exhibited remarkable performance in solving visionrelated problems, especially in unpredictable, dynamic, and challenging environments. In autonomous vehicles, imitation-learning-based steering angle prediction is viable due to the visual imagery comprehension of CNNs. In this regard, globally, researchers are currently focusing on the architectural design and optimization of the hyperparameters of CNNs to achieve the best results. Literature has proven the superiority of metaheuristic algorithms over the manual-tuning of CNNs. However, to the best of our knowledge, these techniques are yet to be applied to address the problem of imitationlearning-based steering angle prediction. Thus, in this study, we examine the application of the bat algorithm and particle swarm optimization algorithm for the optimization of the CNN model and its hyperparameters, which are employed to solve the steering angle prediction problem. To validate the performance of each hyperparameters' set and architectural parameters' set, we utilized the Udacity steering angle dataset and obtained the best results at the following hyperparameter set: optimizer, Adagrad; learning rate, 0.0052; and nonlinear activation function, exponential linear unit. As per our findings, we determined that the deep learning models show better results but require more training epochs and time as compared to shallower ones. Results show the superiority of our approach in optimizing CNNs through metaheuristic algorithms as compared with the manual-tuning approach. Infield testing was also performed using the model trained with the optimal architecture, which we developed using our approach.

show abstract

“…There are different approaches to solve HPO tasks in ML, the most common being various grid and random search approaches [35,36]. However, various metaheuristic algorithms have shown some very good results when optimizing hyper-parameters in deep neural networks recently [14,37,38].…”

Section: Hyper-parameter Optimization In Machine Learningmentioning

confidence: 99%

Classification of Similar Sports Images Using Convolutional Neural Network with Hyper-Parameter Optimization

2020

View full text Add to dashboard Cite

With the exponential growth of the presence of sport in the media, the need for effective classification of sports images has become crucial. The traditional approaches require carefully hand-crafted features, which make them impractical for massive-scale data and less accurate in distinguishing images that are very similar in appearance. As the deep learning methods can automatically extract deep representation of training data and have achieved impressive performance in image classification, our goal was to apply them to automatic classification of very similar sports disciplines. For this purpose, we developed a CNN-TL-DE method for image classification using the fine-tuning of transfer learning for training a convolutional neural network model with the use of hyper-parameter optimization based on differential evolution. Through the automatic optimization of neural network topology and essential training parameters, we significantly improved the classification performance evaluated on a dataset composed from images of four similar sports—American football, rugby, soccer, and field hockey. The analysis of interpretable representation of the trained model additionally revealed interesting insights into how our model perceives images which contributed to a greater confidence in the model prediction. The performed experiments showed our proposed method to be a very competitive image classification method for distinguishing very similar sports and sport situations.

show abstract

Using Metaheuristics for Hyper-Parameter Optimization of Convolutional Neural Networks

Cited by 25 publications

References 9 publications

Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor

Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor

Optimizing Steering Angle Predictive Convolutional Neural Network for Autonomous Car

Classification of Similar Sports Images Using Convolutional Neural Network with Hyper-Parameter Optimization

Contact Info

Product

Resources

About