An Evolutionary Deep Learning Approach Using Genetic Programming with Convolution Operators for Image Classification

Abstract

Evolutionary deep learning (EDL) as a hot topic in recent years aims at using evolutionary computation (EC) techniques to address existing issues in deep learning. Most existing work focuses on employing EC methods for evolving hyper-parameters, deep structures or weights for neural networks (NNs). Genetic programming (GP) as an EC method is able to achieve deep learning due to the characteristics of its representation. However, many current GP-based EDL methods are limited to binary image classification. This paper proposed a new GP-based EDL method with convolution operators (COGP) for feature learning on binary and multi-class image classification. A novel flexible program structure is developed to allow COGP to evolve solutions with deep or shallow structures. Associated with the program structure, a new function set and a new terminal set are developed in COGP. The experimental results on six different image classification data sets of varying difficulty demonstrated that COGP achieved significantly better performance in most comparisons with 11 effectively competitive methods. The visualisation of the best program further revealed the high interpretability of the solutions found by COGP.

Genetic Programming-Based Feature Learning for Facial Expression Classification

“…This method employed a number of image filters and max-pooling operators as functions so that domain-specific features can be extracted from images. Bi et al [9] developed a GP-based method with convolution operators for feature learning. The filters and the size of filters can be automatically selected by this GP method, which is more flexible than that in CNNs.…”

“…Instead of manually extracting features, many methods have been developed to automatically extract features from images for classification [5]. Typical methods are convolutional neural networks (CNNs) and genetic programming (GP) [9,10]. In these methods, features are automatically learned/extracted and then classification is performed using these features on a training set.…”

“…Thus, these methods are more effective for different image classification tasks than the methods using manually extracted features [11]. However, most of the existing methods are neural network (NN)-based methods, which often require a large amount of training data [9,10]. Therefore, this paper aims to develop a non-NN-based method for facial expression classification, even using a small number of training images.…”

“…In recent years, GP has been widely applied to image classification [10]. Existing works have shown that GP can extract effective and domain-specific features for object classification, texture image classification, and scene classification [9,11,14,15]. However, very few GP-based methods have been developed for facial expression classification.…”

“…However, very few GP-based methods have been developed for facial expression classification. The methods such as [9,11,14] could be used for facial expression classification, but they may not be effective and efficient because they were developed for different tasks. The features effective for texture classification may not be effective for facial expression classification.…”