A Neural Network pruning approach based on Compressive Sampling

Yang, Jie; Bouzerdoum, A.; Phung, Son Lam

doi:10.1109/ijcnn.2009.5179045

Cited by 10 publications

(6 citation statements)

References 17 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Before we formulate the problem of network pruning as a compressive sampling problem we introduce some definitions [11,10] We assume that the training input patterns are stored in a matrix I, and the desired output patterns are stored in a matrix O, then the mathematical model for training of the neural network can be extracted in the form of the following expansion: w . So we can write problem as below: …”

Section: Problem Formulation and Methodologymentioning

confidence: 99%

Sound Source Localization with CS Based Compressed Neural Network

Dehkordi¹

2012

AJSP

View full text Add to dashboard Cite

Microphone arrays are today employed to specify the sound source locations in numerous real time applications such as speech processing in large rooms or acoustic echo cancellation. Signal sources may exist in the near field or far field with respect to the microphones. Current Neural Networks (NNs) based source localization approaches assume far field narrowband sources. One of the important limitations of these NN-based approaches is making balance between computational complexity and the development of NNs; an architecture that is too large or too small will affect the performance in terms of generalization and computational cost. In the previous analysis, saliency subject has been employed to determine the most suitable structure, however, it is time-consuming and the performance is not robust. In this paper, a family of new algorithms for compression of NNs is presented based on Compressive Sampling (CS) theory. The proposed framework makes it possible to find a sparse structure for NNs, and then the designed neural network is compressed by using CS. The key difference between our algorithm and the state-of-the-art techniques is that the mapping is continuously done using the most effective features; therefore, the proposed method has a fast convergence. The empirical work demonstrates that the proposed algorithm is an effective alternative to traditional methods in terms of accuracy and computational complexity.

show abstract

Section: Problem Formulation and Methodologymentioning

confidence: 99%

Sound Source Localization with CS Based Compressed Neural Network

Dehkordi¹

2012

AJSP

View full text Add to dashboard Cite

show abstract

“…This section briefly surveys the sparse representation and multiple measure vector (MMV) model [17], [18]. Given an original signal and a dictionary consisting of basis vectors, the sparse representation addresses a compact approximation for the given signal using basis vectors.…”

Section: Sparse Representationmentioning

confidence: 99%

A structure optimization algorithm of neural networks for large-scale data sets

Yang

Berryman

et al. 2014

2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)

Self Cite

View full text Add to dashboard Cite

(2014). A structure optimization algorithm of neural networks for large-scale data sets. 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (pp. 956-961). United States: IEEE.A structure optimization algorithm of neural networks for large-scale data sets AbstractOver the past several decades, neural networks have evolved into powerful computation systems, which are able to learn complex nonlinear input-output relationship from data. However, the structure optimization problem of neural network is a big challenge for processing huge-volumed, diversified and uncertain data. This paper focuses on this problem and introduces a network pruning algorithm based on sparse representation, termed SRP. The proposed approach starts with a large network, then selects important hidden neurons from the original structure using a forward selection criterion that minimizes the residual output error. Furthermore, the presented algorithm has no constraints on the network type. The efficiency of the proposed approach is evaluated based on several benchmark data sets. We also evaluate the performance of the proposed algorithm on a real-world application of individual travel mode choice. Abstract-Over the past several decades, neural networks have evolved into powerful computation systems, which are able to learn complex nonlinear input-output relationship from data. However, the structure optimization problem of neural network is a big challenge for processing huge-volumed, diversified and uncertain data. This paper focuses on this problem and introduces a network pruning algorithm based on sparse representation, termed SRP. The proposed approach starts with a large network, then selects important hidden neurons from the original structure using a forward selection criterion that minimizes the residual output error. Furthermore, the presented algorithm has no constraints on the network type. The efficiency of the proposed approach is evaluated based on several benchmark data sets. We also evaluate the performance of the proposed algorithm on a real-world application of individual travel mode choice. The experimental results have shown that SRP performs favorably compared to alternative approaches.

show abstract

“…In addition, Yang et al [24] presented a similar framework for network pruning and is described as follows:…”

Section: Neural Network Pruningmentioning

confidence: 99%

Optimizing Artificial Neural Networks using Cat Swarm Optimization Algorithm

Yusiong¹

2012

IJISA

View full text Add to dashboard Cite

Abstract-An Artificial Neural Network (ANN) is an abstract representation of the biological nervous system which has the ability to solve many complex problems. The interesting attributes it exhibits makes an ANN capable of -learning‖. ANN learning is achieved by training the neural network using a training algorithm. Aside from choosing a training algorithm to train ANNs, the ANN structure can also be optimized by applying certain pruning techniques to reduce network complexity. The Cat Swarm Optimization (CSO) algorithm, a swarm intelligence-based optimization algorithm mimics the behavior of cats, is used as the training algorithm and the Optimal Brain Damage (OBD) method as the pruning algorithm. This study suggests an approach to ANN training through the simultaneous optimization of the connection weights and ANN structure. Experiments performed on benchmark datasets taken from the UCI machine learning repository show that the proposed CSONN-OBD is an effective tool for training neural networks.

show abstract

A Neural Network pruning approach based on Compressive Sampling

Cited by 10 publications

References 17 publications

Sound Source Localization with CS Based Compressed Neural Network

Sound Source Localization with CS Based Compressed Neural Network

A structure optimization algorithm of neural networks for large-scale data sets

Optimizing Artificial Neural Networks using Cat Swarm Optimization Algorithm

Contact Info

Product

Resources

About