Improving Bag-of-Deep-Visual-Words Model via Combining Deep Features With Feature Difference Vectors

Abstract

Bag-of-Deep-Visual-Words (BoDVW) model has shown its advantage to Convolutional Neural Network (CNN) model in image classification tasks with a small number of training samples. An essential step in BoDVW model is to extract deep features by using an off-the-shelf CNN model as a feature extractor. Two deep feature extraction methods have been raised in recent years. The first method densely samples multi-scale image patches and then converts them into deep features via a deep-level fullyconnected layer.The second method uses the output of a deep-level convolutional layer or pooling layer as the source of deep features. By contrast, the second method is much more efficient. However, it performs worse than the first method in classification accuracy. The reason is that deep features extracted by the second method are yielded in receptive fields of a single size. To make BoDVW model have high feature extraction efficiency and high classification accuracy, we propose enhancing deep features extracted by the second method at low added computation costs by supplementing the information obtained from receptive fields of different sizes. Concretely, we raise a novel feature named "feature difference (FD) vector" in this article. It can roughly preserve the information of multiple deep features extracted by the convolutional layers of different receptive field sizes. Each deep feature is enhanced by combining an FD vector to form a combined feature. The image representation vector of an image is generated using the combined features extracted from it. Our experimental results on three public datasets (15-Scenes, TF-Flowers, and NWPU-RESISC45) show that our method can avoid the high computation costs of the first method and achieve comparable results to the first method, which exhibits the effectiveness of our method.

Comparison of estimating vegetation index for outdoor free-range pig production using convolutional neural networks

“…The first winner, AlexNet [ 16 ], expanded the input image size from 32 × 32 in LeNet to 224 × 224, increasing the model size, but solved the potential problem of overfitting by applying dropout layers and significantly improved its accuracy from 73.8% to 83.7% by applying the rectified linear unit (ReLU) activation function to the ImageNet tests. VggNet [ 17 ] achieved remarkable results with an accuracy of 93.2% by increasing the number of convolutional filters and expanding the layer structure while unifying the convolution filter size to 3 × 3 to reduce computation. However, there was no significant improvement compared to having 16 layers.…”

Can using a pre-trained deep learning model as the feature extractor in the bag-of-deep-visual-words model always improve image classification accuracy?

“…The other (denoted as Ext-DFs(FC)) extracts multi-scale image patches and uses the output vectors from a fully-connected or global pooling layer as deep features. A recent study [ 41 ] utilized these two methods, but it did not provide experimental results under different dictionary sizes and encoding methods, and also did not fully analyze the computational costs. In this article, we take a closer look at these two methods.…”