A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Abstract

Recent advances in Convolutional Neural Networks (CNNs) have obtained promising results in difficult deep learning tasks. However, the success of a CNN depends on finding an architecture to fit a given problem. A hand-crafted architecture is a challenging, time-consuming process that requires expert knowledge and effort, due to a large number of architectural design choices. In this article, we present an efficient framework that automatically designs a high-performing CNN architecture for a given problem. In this framework, we introduce a new optimization objective function that combines the error rate and the information learnt by a set of feature maps using deconvolutional networks (deconvnet). The new objective function allows the hyperparameters of the CNN architecture to be optimized in a way that enhances the performance by guiding the CNN through better visualization of learnt features via deconvnet. The actual optimization of the objective function is carried out via the Nelder-Mead Method (NMM). Further, our new objective function results in much faster convergence towards a better architecture. The proposed framework has the ability to explore a CNN architecture's numerous design choices in an efficient way and also allows effective, distributed execution and synchronization via web services. Empirically, we demonstrate that the CNN architecture designed with our approach outperforms several existing approaches in terms of its error rate. Our results are also competitive with state-of-the-art results on the MNIST dataset and perform reasonably against the state-of-the-art results on CIFAR-10 and CIFAR-100 datasets. Our approach has a significant role in increasing the depth, reducing the size of strides, and constraining some convolutional layers not followed by pooling layers in order to find a CNN architecture that produces a high recognition performance.

A Deep-Learning Approach for Diagnosis of Metastatic Breast Cancer in Bones from Whole-Body Scans

“…Due to the fact that there are various remains and non-osseous uptake presented in images (i.e., urine contamination and medical accessories), as well as the frequent visible site of radiopharmaceutical injection [77], a preprocessing approach was accomplished to remove these artifacts from the original images. This preprocessing method was accomplished by a nuclear medicine physician, before the use of the dataset in the proposed classification approach.…”

Deep Architectures for Human Activity Recognition using Sensors

“…The first convolution layer identifies low level features whereas next convolutional layers detect higher level features (Namatēvs, 2017). The convolutional layers then introduce nonlinearities to the model through using activation functions such as tanh, sigmoid and rectified linear unit (ReLU) (Albelwi & Mahmood, 2017). Pooling layer is used to downsample the dimensionality of the feature map.It compresses features and reduces network's computational complexity (Affonso, Rossi, Vieira & Ferreira, 2017).…”

Deep Convolutional Neural Networks: Structure, Feature Extraction and Training

“…The standard way to model a neuron's output f as a function of its input x is with f(x) = tanh(x), sigmoid(x) or Rectified Linear Unit (ReLU) [32]. The last one is preferable because it makes training several times faster than its equivalents.…”

“…From the above-mentioned considerations, it can be concluded that a new feature map is typically generated by sliding a filter over the input and computing the dot product (which is similar to the convolution operation), followed by a non-linear activation function to introduce non-linearity into the model [32].…”

“…The latter consists of a small number of fully-connected layers, but the final layer is often a sofmax classifier [32]. Accordingly with a complex layer terminology, one convolutional net or convolutional layer is composed of convolutional stage (e.g., affine transformation), detector stage (e.g., rectified linear), and pooling stage [1].…”