Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification

Sun, Yanan; Xue, Bing; Zhang, Mengjie; Yen, Gary G.; Lv, Jiancheng

doi:10.1109/tcyb.2020.2983860

Cited by 622 publications

(355 citation statements)

References 45 publications

(96 reference statements)

Supporting

Mentioning

300

Contrasting

Unclassified

Order By: Relevance

“…However, most real-world users often do not have such knowledge. In recent years, evolutionary computation (EC) has shown to be effective in automatically searching for the optimal architecture of CNNs [5] [6] [7].…”

Section: Introductionmentioning

confidence: 99%

A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

Wang

Sun

Xue

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy.

show abstract

Section: Introductionmentioning

confidence: 99%

A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

Wang

Sun

Xue

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, deep neural networks (DNNs) [1] have performed excellently in machine learning tasks such as recognition [2,3] and pattern analysis [4]. Particularly, the genetic algorithm [5] and the ant colony algorithm [6] using bio-inspired methods [7] have shown improved image recognition results. The genetic algorithm can find the most optimally performing convolutional neural network (CNN) structure among given CNN models for an image task without pre-or post-processing.…”

Section: Introductionmentioning

confidence: 99%

Selective Poisoning Attack on Deep Neural Networks †

2019

View full text Add to dashboard Cite

Studies related to pattern recognition and visualization using computer technology have been introduced. In particular, deep neural networks (DNNs) provide good performance for image, speech, and pattern recognition. However, a poisoning attack is a serious threat to a DNN’s security. A poisoning attack reduces the accuracy of a DNN by adding malicious training data during the training process. In some situations, it may be necessary to drop a specifically chosen class of accuracy from the model. For example, if an attacker specifically disallows nuclear facilities to be selectively recognized, it may be necessary to intentionally prevent unmanned aerial vehicles from correctly recognizing nuclear-related facilities. In this paper, we propose a selective poisoning attack that reduces the accuracy of only the chosen class in the model. The proposed method achieves this by training malicious data corresponding to only the chosen class while maintaining the accuracy of the remaining classes. For the experiment, we used tensorflow as the machine-learning library as well as MNIST, Fashion-MNIST, and CIFAR10 as the datasets. Experimental results show that the proposed method can reduce the accuracy of the chosen class by 43.2%, 41.7%, and 55.3% in MNIST, Fashion-MNIST, and CIFAR10, respectively, while maintaining the accuracy of the remaining classes.

show abstract

“…El entrenamiento de una CNN requiere mucho tiempo y es computacionalmente costoso pues utiliza grandes conjuntos de imágenes de entrenamiento (típicamente se requieren varias decenas de miles de imágenes). Esto dificulta el empleo de otras técnicas que buscan mejorar el desempeño de CNNs o lograr el diseño automático de redes; por ejemplo, los trabajos [6] y [7] proponen llevar a cabo la optimización de la arquitectura de CNNs para las tareas de clasificación de imágenes (ver Fig. 1), mediante el uso de algoritmos evolutivos.…”

Section: Introductionunclassified

“…Aunque dicha estrategia de optimización de la arquitectura de CNN mediante un algoritmo evolutivo ha resultado exitosa, su complejidad computacional es muy elevada y puede ser prohibitiva para muchos investigadores y practicantes que no cuentan con una infraestructura de cómputo suficientemente potente. Por ejemplo, en [7] se menciona que el tiempo requerido para la evolución de una CNN utilizando una variante de algoritmo genético y aplicada al conjunto de datos CIFAR10 1 fue de 35 días de cómputo en una GPU; mientras, en [8] se reporta que el método allí descrito consume 2,750 días de cómputo en una GPU para evolucionar una red para el mismo problema de CIFAR10 (para más datos de sistemas del estado del arte, ver [7]).…”

Section: Introductionunclassified

See 1 more Smart Citation

Entrenamiento dinámico de redes convolucionales profundas para clasificación de imágenes

Frausto-Pérez¹,

Rojas-Domínguez²,

Ornelas-Rodríguez³

et al. 2019

RCS

View full text Add to dashboard Cite

Resumen. En años recientes, el Aprendizaje Profundo se ha posicionado como uno de los paradigmas más exitosos para la solución de problemas de reconocimiento de patrones, tales como clasificación de imágenes. En particular, las CNNs (Convolutional Neural Networks) constituyen uno de los modelos más utilizados para esta tarea. Una CNN típica incorpora cientos de miles de parámetros que deben ser ajustados iterativamente, en lo que se conoce como el entrenamiento de la red, mediante el uso de grandes bancos de imágenes de referencia. Este entrenamiento representa un alto costo de recursos y tiempo computacional incluso cuando se cuenta con GPUs para realizar el procesamiento. En este trabajo se propone un esquema alternativo p ara el manejo de los datos de entrenamiento para CNNs, que consiste en un muestreo selectivoadaptativo de los datos. M ediante experimentos con la base de datos CIFAR10 para clasificación de imágenes, se demuestra que este esquema consigue disminuir el tiempo de entrenamiento sin sacrificar significativamente el desempeño de las redes.Palabras clave: redes neuronales convolucionales, clasificación de imágenes , tiempo de entrenamiento.Abstract. In recent years, Deep Learning has positioned itself as one of the most successful paradigms for the solution of pattern recognition problems such as image classification. Particularly, CNNs (Convolutional Neural Networks) are one of the most used models for this task. A typical CNN incorporates hundreds of thousands of parameters that must be adjusted in what is known as t he training of the network, through the use of large reference training sets. This training represents a high cost in computational time and resources even with the availability of GPUs to do the processing. In this work, an alternative scheme for management of the training data for CNNs is proposed which consists in the selective-adaptive sampling of the data. Through experiments performed on the CIFAR10 dataset for image classification, it is shown that the proposed scheme succeeds in decreasing the training time without significantly sacrificing the performance of the networks.

show abstract

Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification

Cited by 622 publications

References 45 publications

A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

Selective Poisoning Attack on Deep Neural Networks †

Entrenamiento dinámico de redes convolucionales profundas para clasificación de imágenes

Contact Info

Product

Resources

About