Neural  Network Structure Optimization Algorithm

Nowakowski, Grzegorz; Dorogyy, Yaroslaw; Doroga-Ivaniuk, Olena O.

doi:10.14313/jamris_1-2018/1

JAMRIS

2018

DOI: 10.14313/jamris_1-2018/1

|View full text |Cite

Neural Network Structure Optimization Algorithm

Grzegorz Nowakowski¹,

Yaroslaw Dorogyy²,

Olena O. Doroga-Ivaniuk³

Abstract: This paper presents a deep analysis of literature on the problems of optimization of parameters and structure of the neural networks and the basic disadvantages that are present in the observed algorithms and methods. As a result, there is suggested a new algorithm for neural network structure optimization, which is free of the major shortcomings of other algorithms. The paper describes a detailed description of the algorithm, its implementation and application for recognition problems.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2021

2022

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 22 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

The Method of Structural Adjustment of Neural Network Models to Ensure Interpretation

Leoshchenko¹,

Oliinyk²,

Субботин³

et al. 2021

RIC

View full text Add to dashboard Cite

Context. The problem of structural modification of pre-synthesized models based on artificial neural networks to ensure the property of interpretation when working with big data is considered. The object of the study is the process of structural modification of artificial neural networks using adaptive mechanisms. Objective of the work is to develop a method for structural modification of neural networks to increase their speed and reduce resource consumption when processing big data. Method. A method of structural adjustment of neural networks based on adaptive mechanisms borrowed from neuroevolutionary synthesis methods is proposed. At the beginning, the method uses a system of indicators to evaluate the existing structure of an artificial neural network. The assessment is based on the structural features of neuromodels. Then the obtained indicator estimates are compared with the criteria values for choosing the type of structural changes. Variants of mutational changes from the group of methods of neuroevolutionary modification of the topology and weights of the neural network are used as variants of structural change. The method allows to reduce the resource intensity during the operation of neuromodels, by accelerating the processing of big data, which expands the field of practical application of artificial neural networks. Results. The developed method is implemented and investigated by the example of using a recurrent artificial network of the long short-term memory type when solving the classification problem. The use of the developed method allowed speed up of the neuromodel with a test sample by 25.05%, depending on the computing resources used. Conclusions. The conducted experiments confirmed the operability of the proposed mathematical software and allow us to recommend it for use in practice in the structural adjustment of pre-synthesized neuromodels for further solving problems of diagnosis, forecasting, evaluation and pattern recognition using big data. The prospects for further research may consist in a more fine-tuning of the indicator system to determine the connections encoding noisy data in order to further improve the accuracy of models based on neural networks.

show abstract

The Method of Structural Adjustment of Neural Network Models to Ensure Interpretation

Leoshchenko¹,

Oliinyk²,

Субботин³

et al. 2021

RIC

View full text Add to dashboard Cite

show abstract

Neural Network Structure Optimization by Simulated Annealing

Kuo

Kuruoğlu

Chan

2022

Entropy

View full text Add to dashboard Cite

A critical problem in large neural networks is over parameterization with a large number of weight parameters, which limits their use on edge devices due to prohibitive computational power and memory/storage requirements. To make neural networks more practical on edge devices and real-time industrial applications, they need to be compressed in advance. Since edge devices cannot train or access trained networks when internet resources are scarce, the preloading of smaller networks is essential. Various works in the literature have shown that the redundant branches can be pruned strategically in a fully connected network without sacrificing the performance significantly. However, majority of these methodologies need high computational resources to integrate weight training via the back-propagation algorithm during the process of network compression. In this work, we draw attention to the optimization of the network structure for preserving performance despite compression by pruning aggressively. The structure optimization is performed using the simulated annealing algorithm only, without utilizing back-propagation for branch weight training. Being a heuristic-based, non-convex optimization method, simulated annealing provides a globally near-optimal solution to this NP-hard problem for a given percentage of branch pruning. Our simulation results have shown that simulated annealing can significantly reduce the complexity of a fully connected network while maintaining the performance without the help of back-propagation.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Neural Network Structure Optimization Algorithm

Cited by 2 publications

References 22 publications

The Method of Structural Adjustment of Neural Network Models to Ensure Interpretation

The Method of Structural Adjustment of Neural Network Models to Ensure Interpretation

Neural Network Structure Optimization by Simulated Annealing

Contact Info

Product

Resources

About