Feeding Hand-Crafted Features for Enhancing the Performance of Convolutional Neural Networks

Abstract

Since the convolutional neural network (CNN) is believed to find right features for a given problem, the study of hand-crafted features is somewhat neglected these days. In this paper, we show that finding an appropriate feature for the given problem may be still important as they can enhance the performance of CNN-based algorithms. Specifically, we show that feeding an appropriate feature to the CNN enhances its performance in some face related works such as age/gender estimation, face detection and emotion recognition. We use Gabor filter bank responses for these tasks, feeding them to the CNN along with the input image. The stack of image and Gabor responses can be fed to the CNN as a tensor input, or as a fused image which is a weighted sum of image and Gabor responses. The Gabor filter parameters can also be tuned depending on the given problem, for increasing the performance. From the extensive experiments, it is shown that the proposed methods provide better performance than the conventional CNN-based methods that use only the input images.

Crop Lodging Prediction From UAV-Acquired Images of Wheat and Canola Using a DCNN Augmented With Handcrafted Texture Features

“…In this research, we design an architecture based on handcrafted and DCNN features. Combining these types of features have been proposed previously [5,12,21]. Using an image along with Gabor filters extracted from that image as input to the network, Hosseini et al [5] achieved a higher accuracy compared to several traditional and CNNbased models.…”

“…Combining these types of features have been proposed previously [5,12,21]. Using an image along with Gabor filters extracted from that image as input to the network, Hosseini et al [5] achieved a higher accuracy compared to several traditional and CNNbased models. Wang et al [21], using a cascaded approach based on combining a CNN model and handcrafted features, proposed a computationally efficient model for counting the number of cells undergoing mitosis.…”

Fusion of Hand-crafted and Deep Features for Automatic Diabetic Foot Ulcer Classification

“…Interestingly, in recent years the researchers' focus has been shifted to combine handcrafted features with deep features which have shown impressive performance in several computer vision tasks and applications such as face detection [23], facial expression recognition [23], age estimation [23], and image recognition [24]. In other words, the models that were trained on the fused features from both the handcrafted and deep features achieved significantly higher accuracy than those trained on the handcrafted features only or deep features only as in [22], [23], [24], [25], [26], [27]. Therefore, in this study, we propose to use the fused features from both the handcrafted and deep features for the recognition of DFU with the presence of ischaemia and infection.…”

Diabetic foot ulcer classification using mapped binary patterns and convolutional neural networks

“…Interestingly, in a recent performance evaluation for several other computer vision tasks [6,24,30], the CNNs models trained on the handcrafted features only or in combination with the RGB images were shown to achieve competitive performance than the CNNs models trained on the RGB images only. For example, an ensemble of CNN models trained on RGB and LBP mapped coded images for emotion recognition [30], texture recognition [6], and remote sensing scene classification [6].…”

“…For example, an ensemble of CNN models trained on RGB and LBP mapped coded images for emotion recognition [30], texture recognition [6], and remote sensing scene classification [6]. In addition, [24] used a weighted sum of RGB images and Gabor responses images is fed the CNN for age estimation, gender classification, face detection, and facial expression recognition. Inspired by these research, and with the great recent success of deep learning and the importance of the texture features in medical imaging, we propose to combine the importance of texture coded images within the deep learning framework to investigate its potential in DFU classification.…”