Analysis of instance selection algorithms on large datasets with Deep Convolutional Neural Networks

Albelwi, Saleh; Mahmood, Ausif

doi:10.1109/lisat.2016.7494142

Cited by 8 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many instance selection algorithms have been proposed and reviewed in [59]. Albelwi and Mahmood [60] evaluated and analyzed the performance of different instance selection algorithms on CNNs. In this framework, for very large datasets, we employ instance selection based on Random Mutual Hill Climbing (RMHC) [61] as a preprocessing step to select the training sample (T S ) which will be used during the exploration phase to find a high-performing architecture.…”

Section: Reducing the Training Setmentioning

confidence: 99%

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Albelwi

Mahmood

2017

Entropy

Self Cite

192

View full text Add to dashboard Cite

Recent advances in Convolutional Neural Networks (CNNs) have obtained promising results in difficult deep learning tasks. However, the success of a CNN depends on finding an architecture to fit a given problem. A hand-crafted architecture is a challenging, time-consuming process that requires expert knowledge and effort, due to a large number of architectural design choices. In this article, we present an efficient framework that automatically designs a high-performing CNN architecture for a given problem. In this framework, we introduce a new optimization objective function that combines the error rate and the information learnt by a set of feature maps using deconvolutional networks (deconvnet). The new objective function allows the hyperparameters of the CNN architecture to be optimized in a way that enhances the performance by guiding the CNN through better visualization of learnt features via deconvnet. The actual optimization of the objective function is carried out via the Nelder-Mead Method (NMM). Further, our new objective function results in much faster convergence towards a better architecture. The proposed framework has the ability to explore a CNN architecture's numerous design choices in an efficient way and also allows effective, distributed execution and synchronization via web services. Empirically, we demonstrate that the CNN architecture designed with our approach outperforms several existing approaches in terms of its error rate. Our results are also competitive with state-of-the-art results on the MNIST dataset and perform reasonably against the state-of-the-art results on CIFAR-10 and CIFAR-100 datasets. Our approach has a significant role in increasing the depth, reducing the size of strides, and constraining some convolutional layers not followed by pooling layers in order to find a CNN architecture that produces a high recognition performance.

show abstract

Section: Reducing the Training Setmentioning

confidence: 99%

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Albelwi

Mahmood

2017

Entropy

Self Cite

192

View full text Add to dashboard Cite

show abstract

“…Not only must a hyperparameter optimization algorithm optimize over variables which are discrete, ordinal, and continuous, but it must simultaneously choose which variables to optimize -a di cult task. Currently, no work covers optimization of every hyperparameter in designing a CNN architecture [1]. We briefly refer in the following to some recent CNN hyperparameter optimization results.…”

Section: Related Work: Methods For Cnn Hyperparameter Optimizationmentioning

confidence: 99%

Weighted Random Search for CNN Hyperparameter Optimization

Andonie

Florea

2020

INT J COMPUT COMMUN, Int. J. Comput. Commun. Control

View full text Add to dashboard Cite

Nearly all model algorithms used in machine learning use two di erent sets of parameters: the training parameters and the meta-parameters (hyperparameters). While the training parameters are learned during the training phase, the values of the hyperparameters have to be specified before learning starts. For a given dataset, we would like to find the optimal combination of hyperparameter values, in a reasonable amount of time. This is a challenging task because of its computational complexity. In previous work [11], we introduced the Weighted Random Search (WRS) method, a combination of Random Search (RS) and probabilistic greedy heuristic. In the current paper, we compare the WRS method with several state-of-the art hyperparameter optimization methods with respect to Convolutional Neural Network (CNN) hyperparameter optimization. The criterion is the classification accuracy achieved within the same number of tested combinations of hyperparameter values. According to our experiments, the WRS algorithm outperforms the other methods.

show abstract

“…The NMA was used in Convolutional Neural Network (CNN) optimization in [1,2], in conjunction with a relatively small optimization dataset. It works well for objective functions that are smooth, unimodal and not too noisy.…”

Section: Derivative-free Optimization: Nelder-meadmentioning

confidence: 99%

“…This may also reduce the search space (relates to A2). For instance, superior results were obtained by combining accuracy with visualization via a deconvolution network [1].…”

Section: Computational Complexity Issuesmentioning

confidence: 99%

“…Some recent P systems are adaptive [13,55], and [3]. During the Brainstorming Week on Membrane Computing Sevilla, 2018, ideas about evolving spiking neural (SN) P systems (introduced in [27]) were discussed 1 . A first attempt to use SN P systems in pattern recognition can be found in [49], where SN P systems were reported to outperform back propagation and probabilistic neural networks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hyperparameter optimization in learning systems

Andonie

2019

J Membr Comput

View full text Add to dashboard Cite

While the training parameters of machine learning models are adapted during the training phase, the values of the hyperparameters (or meta-parameters) have to be specified before the learning phase. The goal is to find a set of hyperparameter values which gives us the best model for our data in a reasonable amount of time. We present an integrated view of methods used in hyperparameter optimization of learning systems, with an emphasis on computational complexity aspects. Our thesis is that we should solve a hyperparameter optimization problem using a combination of techniques for: optimization, search space and training time reduction. Case studies from real-world applications illustrate the practical aspects. We create the framework for a future separation between parameters and hyperparameters in adaptive P systems.

show abstract

Analysis of instance selection algorithms on large datasets with Deep Convolutional Neural Networks

Cited by 8 publications

References 14 publications

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Weighted Random Search for CNN Hyperparameter Optimization

Hyperparameter optimization in learning systems

Contact Info

Product

Resources

About