Metaheuristic Algorithms for Convolution Neural Network

Rere, L. M. Rasdi; Fanany, Mohamad Ivan; Arymurthy, Aniati Murni

doi:10.1155/2016/1537325

Cited by 83 publications

(44 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cuckoo search and bat algorithm algorithms were used in [56] to adjust the weights of neural networks of feed-forward type. In [57] an algorithm was designed using metaheuristics to optimize convolutional neural networks. An application to long term short memory (LSTM) neural network training was studied in [58].…”

Section: Hybridizing Metaheuristics With Machine Learningmentioning

confidence: 99%

A Hybrid k-Means Cuckoo Search Algorithm Applied to the Counterfort Retaining Walls Problem

2020

View full text Add to dashboard Cite

The counterfort retaining wall is one of the most frequent structures used in civil engineering. In this structure, optimization of cost and CO2 emissions are important. The first is relevant in the competitiveness and efficiency of the company, the second in environmental impact. From the point of view of computational complexity, the problem is challenging due to the large number of possible combinations in the solution space. In this article, a k-means cuckoo search hybrid algorithm is proposed where the cuckoo search metaheuristic is used as an optimization mechanism in continuous spaces and the unsupervised k-means learning technique to discretize the solutions. A random operator is designed to determine the contribution of the k-means operator in the optimization process. The best values, the averages, and the interquartile ranges of the obtained distributions are compared. The hybrid algorithm was later compared to a version of harmony search that also solved the problem. The results show that the k-mean operator contributes significantly to the quality of the solutions and that our algorithm is highly competitive, surpassing the results obtained by harmony search.

show abstract

Section: Hybridizing Metaheuristics With Machine Learningmentioning

confidence: 99%

A Hybrid k-Means Cuckoo Search Algorithm Applied to the Counterfort Retaining Walls Problem

2020

View full text Add to dashboard Cite

show abstract

“…Likely the most well-known algorithm in the field of neuroevolution is the neuro-evolution of augmented topologies algorithm (NEAT) [13]. Recent studies have applied evolutionary strategies to deep neural networks in a variety of manners, such as evolving network weights via genetic programming [9,15], differential evolution [11], pitting networks against each other in a tournament selection environment [10], more recent extensions of NEAT [3,7], and by evolving a network's activation functions [4]. The general "mood" of the intersection of these two fields is very much an exploratory one.…”

Section: Related Workmentioning

confidence: 99%

Evolutionary Construction of Convolutional Neural Networks

Knippenberg

Menkovski

Consoli

2019

Machine Learning, Optimization, and Data Science

View full text Add to dashboard Cite

Neuro-Evolution is a field of study that has recently gained significantly increased traction in the deep learning community. It combines deep neural networks and evolutionary algorithms to improve and/or automate the construction of neural networks. Recent Neuro-Evolution approaches have shown promising results, rivaling hand-crafted neural networks in terms of accuracy. A two-step approach is introduced where a convolutional autoencoder is created that efficiently compresses the input data in the first step, and a convolutional neural network is created to classify the compressed data in the second step. The creation of networks in both steps is guided by by an evolutionary process, where new networks are constantly being generated by mutating members of a collection of existing networks. Additionally, a method is introduced that considers the trade-off between compression and information loss of different convolutional autoencoders. This is used to select the optimal convolutional autoencoder from among those evolved to compress the data for the second step. The complete framework is implemented, tested on the popular CIFAR-10 data set, and the results are discussed. Finally, a number of possible directions for future work with this particular framework in mind are considered, including opportunities to improve its efficiency and its application in particular areas.

show abstract

“…However, learning such layers under conditions of a limited size of training datasets and computing resources based on the error backpropagation and the stochastic gradient descent is ineffective. One of the promising ways of training neural networks and fine tuning is the application of metaheuristic algorithms because they are characterized by better convergence and less probability of getting stuck in the "bad" local optimum [17]. Among them, it is worth highlighting the simulated annealing algorithm, since its use made it possible to exceed the results of the gradient descent algorithms in optimization of neural network classifiers [4].…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

Improving the effectiveness of training the on-board object detection system for a compact unmanned aerial vehicle

Москаленко

Dovbysh

Naumenko

et al. 2018

EEJET

View full text Add to dashboard Cite

Запропоновано модель детектора об'єктiв i критерiй ефективностi навчання моделi. Модель мiстить 7 перших модулiв згорткової мережi Squeezenet, два згортковi рiзномасштабнi шари, та iнформацiйно-екстремальний класифiкатор. Як критерiй ефективностi навчання моделi детектора розглядається мультиплiкативна згортка частинних критерiїв, що враховує ефективнiсть виявлення об'єктiв на зображеннi та точнiсть класифiкацiйного аналiзу. При цьому додаткове використання алгоритму ортогонального узгодженого кодування при обчисленнi високорiвневих ознак дозволяє збiльшити точнiсть моделi на 4 %. Розроблено алгоритм навчання детектора об'єктiв за умов малого обсягу розмiчених навчальних зразкiв та обмежених обчислювальних ресурсiв, доступних на борту малогабаритного безпiлотного апарату. Суть алгоритму полягає в адаптацiї верхнiх шарiв моделi до доменної областi використання на основi алгоритмiв зростаючого розрiджено кодуючого нейронного газу та симуляцiї вiдпалу. Навчання верхнiх шарiв без вчителя дозволяє ефективно використати нерозмiченi данi з доменної областi та визначити необхiдну кiлькiсть нейронiв. Показано, що за вiдсутностi тонкої настройки згорткових шарiв забезпечується 69 % виявлених об'єктiв на зображеннях тестової вибiрки Inria Aerial Image. При цьому пiсля тонкої настройки на основi алгоритму симуляцiї вiдпалу забезпечується 95 % виявлених об'єктiв на тестових зображеннях. Показано, що використання попереднього навчання без вчителя дозволяє пiдвищити узагальнюючу здатнiсть вирiшальних правил та прискорити iтерацiйний процес знаходження глобального максимуму при навчаннi з учителем на вибiрцi обмеженого обсягу. При цьому усунення ефекту перенавчання здiйснюється шляхом оптимального вибору значення гiперпараметру, що характеризує ступiнь покриття вхiдних даних нейронами мережi Ключовi слова: зростаючий нейронний газ, детектор об'єктiв, iнформацiйний критерiй, алгоритм симуляцiя вiдпалу

show abstract

Metaheuristic Algorithms for Convolution Neural Network

Cited by 83 publications

References 16 publications

A Hybrid k-Means Cuckoo Search Algorithm Applied to the Counterfort Retaining Walls Problem

A Hybrid k-Means Cuckoo Search Algorithm Applied to the Counterfort Retaining Walls Problem

Evolutionary Construction of Convolutional Neural Networks

Improving the effectiveness of training the on-board object detection system for a compact unmanned aerial vehicle

Contact Info

Product

Resources

About