A Grammar-based Genetic Programming Approach to Optimize Convolutional Neural Network Architectures

Diniz, Jessica Barbosa; Cordeiro, Fernando; Miranda, Péricles; Silva, Laura A. Tomaz Da

doi:10.5753/eniac.2018.4406

Cited by 9 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Os trabalhos a seguir consideraram não apenas a otimização dos parâmetros de DNNs (como os trabalhos anteriores), mas também a composição e a sequência de suas camadas. Diniz et al [Diniz et al 2018] propôs uma abordagem que utiliza Genetic Based Genetic Programming (GGP) para otimizar as arquiteturas de rede neural convolucional (parâmetros e camadas) considerando a acurácia como função objetivo. Miikkulainen et al [Miikkulainen et al 2019] propôs o CoDeepNEAT, uma técnica de neuroevolução para otimizar arquiteturas da DNN.…”

Section: Trabalhos Relacionadośunclassified

Otimização multi-objetivo da sequência de camadas em arquiteturas de aprendizagem profunda

Castro¹,

Neto²,

Miranda³

et al. 2019

Anais Do Encontro Nacional De Inteligência Artificial E Computacional (ENIAC 2019)

View full text Add to dashboard Cite

Selecionar a melhor arquitetura para uma Rede Neural Profunda (DNN) não é uma tarefa trivial, pois há uma enorme quantidade de configurações possíveis (camadas e parâmetros) e grande dificuldade em como escolhê-las. A fim de tornar essa tarefa mais independente da interação humana, este trabalho propõe um método inteligente para otimizar a arquitetura (sequência de camadas) de uma DNN estruturada em cadeia, levando em consideração múltiplos critérios: acurácia e F1 score. O método foi avaliado quanto ao desempenho e comparado às abordagens exaustiva e aleatória. Os resultados obtidos são promissores, mostrando o potencial do método proposto.

show abstract

Section: Trabalhos Relacionadośunclassified

Otimização multi-objetivo da sequência de camadas em arquiteturas de aprendizagem profunda

Castro¹,

Neto²,

Miranda³

et al. 2019

Anais Do Encontro Nacional De Inteligência Artificial E Computacional (ENIAC 2019)

View full text Add to dashboard Cite

show abstract

“…The task of choosing suitable Convolutional Neural Networks (CNNs) and their parameters for a given classification problem is not trivial due to the great variety of algorithms and number of configurable parameters [Diniz et al 2018]. Thus, many works have proposed solutions to facilitate or automate the selection/generation of CNN architectures to help experts make decisions.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, many works have proposed solutions to facilitate or automate the selection/generation of CNN architectures to help experts make decisions. Recently, some works [Assunc ¸ão et al 2018, Diniz et al 2018,de Lima et al 2019,Neto et al 2020,da Silva et al 2021b,da Silva et al 2021a,Lima et al 2022, da Silva et al 2023] have employed Grammatical Evolution (GE) [O'Neill and Ryan 2001] for the generation of CNN architectures. The CNNs produced by GE frameworks have reached promising results, overcoming state-of-the-art CNNs in some image classification problems.…”

Section: Introductionmentioning

confidence: 99%

Assessing Multi-Objective Search Engines for GE: A Case Study in CNN Generation

Chagas,

Rosa,

Silva

et al. 2023

Anais Do XX Encontro Nacional De Inteligência Artificial E Computacional (ENIAC 2023)

View full text Add to dashboard Cite

In recent years, the number of available Convolutional Neural Networks (CNNs) has increased significantly, making it difficult to select an appropriate CNN for a specific problem. To address this challenge, researchers have proposed automated techniques for optimizing CNN architectures, with Grammatical Evolution (GE) being one of the most promising approaches. GE uses context-free grammar to generate programs (e.g., CNNs) and a search engine to find the best solutions. Although several grammars have been proposed for CNN generation, there has been no research evaluating the impact of different search engines in the GE optimization process. This study treats the CNN generation as a multi-objective problem by optimizing accuracy and F1-score, and evaluates seven different multi-objective optimizers listed in the literature as potential search engines. The goal is to investigate the strengths and weaknesses of each optimizer in CNN generation. The experiments were performed on the widely-used CIFAR-10 image classification dataset, and the results showed that selecting the right optimizer for the task is crucial and can have a significant impact on the final result, especially when the number of generations is limited.

show abstract

“…Based on the number of learned features, existing GP-based feature learning methods can be broadly classified into two groups, i.e., learning one feature [13,14,27,138] and learning multiple features [10,16,197]. It is also noted that GP has been applied to evolve CNNs for image classification, such as in [64,206,250], but these methods use CNNs for feature learning rather than GP. Therefore, these methods are not included in any of these two groups.…”

Section: Limitations Of Existing Workmentioning

confidence: 99%

“…Compared with CGP-CNNs, this method has achieved better classification but found models with more parameters. Diniz et al [64] designed grammar-based GP to find the architectures of CNNs for image classification. This method used simple grammar to encode an architecture of CNN, i.e., the convolutional layer, the pooling layer and the fully-connected layer.…”

Section: Evolving Neural Network For Image Classificationmentioning

confidence: 99%

Genetic Programming for Feature Learning in Image Classification

Depositor¹

View full text Add to dashboard Cite

Image classification is an important and fundamental task in computer vision and machine learning. The task is to classify images into one of some pre-defined groups based on the content in the images. However, image classification is a challenging task due to high variations across images, such as illumination, viewpoint, scale variations, deformation, and occlusion. To effectively solve image classification, it is necessary to extract or learn a set of meaningful features from raw pixels or images. The effectiveness of these features significantly affects classification performance. Feature learning aims to automatically learn effective features from images for classification. However, feature learning is difficult due to the high variations of images and the large search space. Genetic Programming (GP) as an Evolutionary Computation (EC) technique is known for its powerful global search ability and high interpretability of the evolved solutions. Compared with other EC methods, GP has a flexible representation of variable length and can search the solution space without any assumptions on the solution structure. The potential of GP in feature learning for image classification has not been comprehensively investigated due to the use of simple representations, e.g., functions and program structures. The overall goal of this thesis is to further investigate and explore the potential of GP for image classification by developing a new GP-based approach with a new representation to automatically learning effective features for different types of image classification tasks. Firstly, this thesis proposes a new GP-based approach with image descriptors to learning global and/or local features for image classification by developing a new program structure, a new function set, a new terminal set, and a new fitness function. These new designs allow GP to detect small regions from the relatively large input image, extract features using image descriptors from the detected regions or the input image, and combine the extracted features for classification. The results show that the new approach significantly outperforms five GP-based methods, eight traditional methods, and three convolutional neural network methods in almost all the comparisons on eight different datasets. Secondly, this thesis proposes a new GP-based approach with a flexible program structure and image-related operators for feature learning in image classification. The new approach learns effective features transformed by multiple layers, i.e., a filtering layer, a pooling layer, a feature extraction layer, and a feature concatenation layer, in a flexible way. The results show that the new approach achieves better performance than a large number of effective methods on 12 benchmark datasets. The solutions and features learned by the new approach provide high interpretability. Thirdly, this thesis proposes the first GP-based approach to automatically and simultaneously learning features and evolving ensembles for image classification. The new approach can learn high-level features through multiple transformations, select effective classification algorithms and optimise the parameters for these classification algorithms to build effective ensembles. The new approach outperforms a large number of benchmark methods on 12 different image classification datasets. Finally, this thesis proposes a multi-population GP-based approach with knowledge transfer and ensembles to improving both the generalisation performance and computational efficiency of GP-based feature learning algorithms for image classification. The new approach can achieve better generalisation performance and computational efficiency than baseline GP-based feature learning method. The new approach can achieve better performance on 11 datasets than a large number of benchmark methods, including many neural network-based methods.

show abstract

A Grammar-based Genetic Programming Approach to Optimize Convolutional Neural Network Architectures

Cited by 9 publications

References 18 publications

Otimização multi-objetivo da sequência de camadas em arquiteturas de aprendizagem profunda

Otimização multi-objetivo da sequência de camadas em arquiteturas de aprendizagem profunda

Assessing Multi-Objective Search Engines for GE: A Case Study in CNN Generation

Genetic Programming for Feature Learning in Image Classification

Contact Info

Product

Resources

About