The effect of choosing optimizer algorithms to improve computer vision tasks: a comparative study

Abstract

Optimization algorithms are used to improve model accuracy. The optimization process undergoes multiple cycles until convergence. A variety of optimization strategies have been developed to overcome the obstacles involved in the learning process. Some of these strategies have been considered in this study to learn more about their complexities. It is crucial to analyse and summarise optimization techniques methodically from a machine learning standpoint since this can provide direction for future work in both machine learning and optimization. The approaches under consideration include the Stochastic Gradient Descent (SGD), Stochastic Optimization Descent with Momentum, Rung Kutta, Adaptive Learning Rate, Root Mean Square Propagation, Adaptive Moment Estimation, Deep Ensembles, Feedback Alignment, Direct Feedback Alignment, Adfactor, AMSGrad, and Gravity. prove the ability of each optimizer applied to machine learning models. Firstly, tests on a skin cancer using the ISIC standard dataset for skin cancer detection were applied using three common optimizers (Adaptive Moment, SGD, and Root Mean Square Propagation) to explore the effect of the algorithms on the skin images. The optimal training results from the analysis indicate that the performance values are enhanced using the Adam optimizer, which achieved 97.30% accuracy. The second dataset is COVIDx CT images, and the results achieved are 99.07% accuracy based on the Adam optimizer. The result indicated that the utilisation of optimizers such as SGD and Adam improved the accuracy in training, testing, and validation stages.

Activation Function Dependence of Data‐Driven Spectra Prediction of Nanostructures

“…The commonly used optimization methods include Stochastic Gradient Descent (SGD), Adaptive Gradient Algorithm (AdaGrad), Root Mean Square Propagation (RMSProp), Adaptive Moment Estimation (Adam), and Nesterov‐accelerated Adaptive Moment Estimation (NAdam). [ 26 ] In our work, NAdam was chosen as our optimization method, which proved to be very optimal and stable.…”

LPBSA: Enhancing Optimization Efficiency through Learner Performance-based Behavior and Simulated Annealing

“…Optimization algorithms play a pivotal role in improving the efficiency of intelligent systems by fine-tuning parameters, reducing computational complexity, and enhancing overall performance. Notably, research by [5] highlights the importance of improving techniques in machine learning and neural network training and accuracy, demonstrating how advancements in optimization algorithms can significantly accelerate convergence and reduce training time. Many real-world problems involve dynamic environments where the optimal solution may change over time, Metaheuristic algorithms with adaptive mechanisms can dynamically adjust their search strategies, enabling them to the scope with changing conditions [6].…”

Benchmarking Variational AutoEncoders on cancer transcriptomics data

“…Despite these promising results, it is known that VAEs suffer from sensitivity to the hyperparameters, such as the learning rate, the number of hidden layers, the optimizer, the number of neurons in each layer [17] [18] [19]. Although VAEs are getting more and more widely-used, there is a lack of guidelines for selecting training hyperparameters.…”