Assessing Multi-Objective Search Engines for GE: A Case Study in CNN Generation

Abstract

In recent years, the number of available Convolutional Neural Networks (CNNs) has increased significantly, making it difficult to select an appropriate CNN for a specific problem. To address this challenge, researchers have proposed automated techniques for optimizing CNN architectures, with Grammatical Evolution (GE) being one of the most promising approaches. GE uses context-free grammar to generate programs (e.g., CNNs) and a search engine to find the best solutions. Although several grammars have been proposed for CNN generation, there has been no research evaluating the impact of different search engines in the GE optimization process. This study treats the CNN generation as a multi-objective problem by optimizing accuracy and F1-score, and evaluates seven different multi-objective optimizers listed in the literature as potential search engines. The goal is to investigate the strengths and weaknesses of each optimizer in CNN generation. The experiments were performed on the widely-used CIFAR-10 image classification dataset, and the results showed that selecting the right optimizer for the task is crucial and can have a significant impact on the final result, especially when the number of generations is limited.