Design of Kernels in Convolutional Neural Networks for Image Classification

Abstract

Abstract. Despite the effectiveness of convolutional neural networks (CNNs) for image classification, our understanding of the effect of shape of convolution kernels on learned representations is limited. In this work, we explore and employ the relationship between shape of kernels which define receptive fields (RFs) in CNNs for learning of feature representations and image classification. For this purpose, we present a feature visualization method for visualization of pixel-wise classification score maps of learned features. Motivated by our experimental results, and observations reported in the literature for modeling of visual systems, we propose a novel design of shape of kernels for learning of representations in CNNs. In the experimental results, the proposed models also outperform the state-of-theart methods employed on the CIFAR-10/100 datasets [1] for image classification. We also achieved an outstanding performance in the classification task, comparing to a base CNN model that introduces more parameters and computational time, using the ILSVRC-2012 dataset [2]. Additionally, we examined the region of interest (ROI) of different models in the classification task and analyzed the robustness of the proposed method to occluded images. Our results indicate the effectiveness of the proposed approach.

Identifying complex motifs in massive omics data with a variable-convolutional layer in deep neural network

“…We noticed that our quantitative model provides an explanation for the observation that some datasets are more sensitive to the kernel length than others, as shown by Sun et al [2016]. Additionally, we showed that our the scoring curve is strongly related to the model's performances.…”

ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks

“…However, when the existing models cannot meet our specific needs, we may not be allowed to customize a new architecture at the costs of heavy human works or numerous GPU hours [36]. Recently, the research community is soliciting innovative architecture-neutral CNN structures, e.g., SE blocks [14] and quasi-hexagonal kernels [30], which can be directly combined with various up-to-date architectures to improve the performance on our real-world applications.…”

Deep Learning-Based X-Ray Baggage Hazardous Object Detection – An FPGA Implementation

“…The study [9] was carried out in computer implementation, not FPGA or embedded system implementation. Ample information on the importance of kernels CNN was provided by Sun et al [10]. In this paper, the authors debated on employing various types of kernels for different types of inputs provided and for multiple applications.…”