Optimization of convolutional neural network using microcanonical annealing algorithm

Ayumi, Vina; Rere, L. M. Rasdi; Fanany, Mohamad Ivan; Arymurthy, Aniati Murni

doi:10.1109/icacsis.2016.7872787

Cited by 43 publications

(27 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…After 8800 iterations, the test error of CNN, CNN with L 2 and PCNN reaches 2.84%, 2.77%, and 2.65%,respectively. [28] 99.22 Graph − CNN [29] 99.14 CNN MA [30] 98.75 MCNN − DS [31] 98.43…”

Section: Mnistmentioning

confidence: 99%

A New Improved Learning Algorithm for Convolutional Neural Networks

Yang

Zhao

et al. 2020

Processes

View full text Add to dashboard Cite

The back-propagation (BP) algorithm is usually used to train convolutional neural networks (CNNs) and has made greater progress in image classification. It updates weights with the gradient descent, and the farther the sample is from the target, the greater the contribution of it to the weight change. However, the influence of samples classified correctly but that are close to the classification boundary is diminished. This paper defines the classification confidence as the degree to which a sample belongs to its correct category, and divides samples of each category into dangerous and safe according to a dynamic classification confidence threshold. Then a new learning algorithm is presented to penalize the loss function with danger samples but not all samples to enable CNN to pay more attention to danger samples and to learn effective information more accurately. The experiment results, carried out on the MNIST dataset and three sub-datasets of CIFAR-10, showed that for the MNIST dataset, the accuracy of Non-improve CNN reached 99.246%, while that of PCNN reached 99.3%; for three sub-datasets of CIFAR-10, the accuracies of Non-improve CNN are 96.15%, 88.93%, and 94.92%, respectively, while those of PCNN are 96.44%, 89.37%, and 95.22%, respectively.

show abstract

Section: Mnistmentioning

confidence: 99%

A New Improved Learning Algorithm for Convolutional Neural Networks

Yang

Zhao

et al. 2020

Processes

View full text Add to dashboard Cite

show abstract

“…Therefore, to use the principles of growing neural gas is promising approach to unsupervised learning of high-level layers and to determine automatically the necessary number of neurons. In this case, the output layers of detector model require fine-tuning, which is typically implemented as one of modifications of the error backpropagation algorithm [2,3]. However, this algorithm is characterized by a low convergence rate and getting stuck in local minima of loss function.…”

Section: Abbreviationsmentioning

confidence: 99%

“…However, this algorithm is characterized by a low convergence rate and getting stuck in local minima of loss function. There are alternative metaheuristic search optimization algorithms, however, effectiveness of using these algorithms in problems of networks fine tuning is scantily explored [3].…”

Section: Abbreviationsmentioning

confidence: 99%

“…Fig. 2 shows a pseudocode of the simulated annealing algorithm, which is implemented by the epochs_max iterations, on each of which function f() is calculated by passing a labeled training dataset through the model of the system of detection and calculation of a complex criterion (2) [3,20]. An analysis of the pseudocode in Fig.…”

Section: Regression Model Is Trained Only On Positive Samples (Matchementioning

confidence: 99%

“…In this case, an initial search point that is formed by the create_initial_solution procedure can be either randomly generated or a result of the preliminarily training by another algorithm. To generate new solutions in the create_neighbor_solution procedure, it is proposed to use the simplest non-adaptive algorithm, which can be represented as formula [3]: The non-maximum suppression algorithm is used for filtering unnecessary actions of the detector to one and the same image object [17,18].…”

Section: Regression Model Is Trained Only On Positive Samples (Matchementioning

confidence: 99%

See 2 more Smart Citations

A Model and Training Algorithm of Small-Sized Object Detection System for a Compact Aerial Drone

Москаленко

Moskalenko

Коробов

et al. 2019

Radio Electronics, Computer Science, Control

View full text Add to dashboard Cite

Context. Lightweight model and effective training algorithm of on-board object detection system for a compact drone are developed. The object of research is the process of small object detection on aerial images under computational resource constraint and uncertainty caused by small amount of labeled training data. The subject of the research are the model and training algorithm for detecting small objects on aerial imagery. Objective. Goal of the research is developing efficient model and training algorithm of object detection system for a compact aerial drone under conditions of restricted computing resources and the limited volume of the labeled training set. Methods. The four-stage training algorithm of the object detector is proposed. At the first stage, selecting the type of deep convolutional neural network and the number of low-level layers that is pre-trained on the ImageNet dataset for reusing takes place. The second stage involves unsupervised training of high-level convolutional sparse coding layers using modification of growing neural gas to automatically determine the required number of neurons and provide optimal distributions of the neurons over the data. Its application makes it possible to utilize the unlabeled training datasets for the adaptation of the high-level feature description to the domain application area. At the third stage, there are reduction of output feture map using principan component analysis and building of decision rules. In this case, output feature map is formed by concatenation of feature maps from different level of deep network using upscaling upper maps to uniform shape for each channel. This approach provides more contextual information for efficient recognition of small objects on aerial images. In order to build classifier of output feature map pixels is proposed to use boosted information-extreme learning. Besides that, the extreme learning machine is used to build of regression model for predict bounding box of detected object. Last stage involves fine-tuning of high-level layers of deep network using simulated annealing metaheuristic algorithm in order to approximating the global optimum of complex criterion of training efficiency of detection model. Results. The results obtained on open datasets testify to the suitability of the model and training algorithm for practical usage. The proposed training algorithm utilize 500 unlabeled and 200 labeled training samples to provide 96% correctly detection of objects on the images of the test dataset. Conclusions. Scientific novelty of the paper is a new model and training algorithm for object detection, which enable to achieve high confidence of recognition of small objects on aerial images under computational resource constraint and limited volume of the labeled training set. Practical significance of the paper results consists in the developed model and training algorithm made it possible to reduce requirements for size of labeled training set and computation resources of on-board detection system of aerial drone in tr...

show abstract

Characterization of abnormalities in breast cancer images using nature‐inspired metaheuristic optimized convolutional neural networks model

Oyelade

Ezugwu

2021

Concurrency and Computation

View full text Add to dashboard Cite

Convolutional neural networks (CNN) are deep learning models widely reported as performing well in several image classification tasks. Training the networks for optimal performance is considered an NP-hard problem. The combination of a metaheuristic algorithm with CNN has been proposed to address this problem because the metaheuristic algorithms have efficient performance, parameter tuning, and stagnation prevention methods. Moreover, there is now an increasing need to apply the best performing metaheuristic algorithm to optimize the parameters, training and learning rate of the CNN models. This study aims to investigate the best performing metaheuristic algorithm for fine-tuning the weights, biases, and hyperparameters of CNN networks for solving the problem of characterization of abnormalities in breast images. Furthermore, hybrid models consisting of a CNN architecture and five representative metaheuristic algorithms to efficiently detect breast cancer abnormalities are presented. The adopted approach involves training a CNN network using genetic algorithm (GA), whale optimization algorithm (WOA), multiverse optimizer (MVO), satin bower optimization (SBO), and life choice-based optimization (LCBO) algorithms to optimize only weights and bias of the model. Two categories of experiments were carried out: the first involved training of the CNN model, while the second involved optimizing the training of CNN model with metaheuristic algorithms. A comparative analysis of the impact of the proposed GA, WOA, MVO, SBO, and LCBO metaheuristic algorithms on the performance of the CNN architecture was carried out. Empirical and statistical analyses are presented to validate further the findings obtained in the study. Results obtained showed that MVO, SBO and LCBO outperformed GA, WOA. The classification accuracy obtained for CNN-GA, CNN-WOA, CNN-MVO, CNN-SBO, and CNN-LCBO at the fifth epoch was 0.76, 0.75, 0.84, 0.80, and 0.86, respectively. At the same time, the traditional CNN achieved an accuracy of 0.66. The outcome of this study showed that contrary to widespread practice, physics-based, biology-based, and human-based optimization algorithms promise better performance parameter tuning for CNN as compared with evolutionary-based and swarm-based metaheuristic algorithms.

show abstract

Optimization of convolutional neural network using microcanonical annealing algorithm

Cited by 43 publications

References 14 publications

A New Improved Learning Algorithm for Convolutional Neural Networks

A New Improved Learning Algorithm for Convolutional Neural Networks

A Model and Training Algorithm of Small-Sized Object Detection System for a Compact Aerial Drone

Characterization of abnormalities in breast cancer images using nature‐inspired metaheuristic optimized convolutional neural networks model

Contact Info

Product

Resources

About