Removal of hidden units and weights for back propagation networks

Hagiwara, Masafumi

doi:10.1109/ijcnn.1993.713929

Cited by 45 publications

(34 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All of the traditional algorithms, such as Optimal Brain Damage (OBD) [16], Optimal Brain Surgeon (OBS) [17], and Magnitude-based pruning (MAG) [18], Skeletonization (SKEL) [6], non-contributing units (NC) [7] and Extended Fourier Amplitude Sensitivity Test (EFAST) [13], are available in SNNS (CSS1 is name of algorithm that uses SMV for sparse representation and CSS2 is another technique that uses MMV for sparse representation). In comparison to other algorithms CSS1 is faster than CSS2 and would achieve smaller computational complexity.…”

Section: Resultsmentioning

confidence: 99%

A compressive sensing based compressed neural network for sound source localization

Dehkordi

Abutalebi

Ghanei

2011

2011 International Symposium on Artificial Intelligence and Signal Processing (AISP)

View full text Add to dashboard Cite

-Microphone arrays are today employed to specify the sound source locations in numerous real time applications such as speech processing in large rooms or acoustic echo cancellation. Signal sources may exist in the near field or far field with respect to the microphones. Current Neural Networks (NNs) based source localization approaches assume far field narrowband sources. One of the important limitations of these NN-based approaches is making balance between computational complexity and the development of NNs; an architecture that is too large or too small will affect the performance in terms of generalization and computational cost. In the previous analysis, saliency subject has been employed to determine the most suitable structure, however, it is time-consuming and the performance is not robust. In this paper, a family of new algorithms for compression of NNs is presented based on Compressive Sampling (CS) theory. The proposed framework makes it possible to find a sparse structure for NNs, and then the designed neural network is compressed by using CS. The key difference between our algorithm and the state-of-the-art techniques is that the mapping is continuously done using the most effective features; therefore, the proposed method has a fast convergence. The empirical work demonstrates that the proposed algorithm is an effective alternative to traditional methods in terms of accuracy and computational complexity.

show abstract

Section: Resultsmentioning

confidence: 99%

A compressive sensing based compressed neural network for sound source localization

Dehkordi

Abutalebi

Ghanei

2011

2011 International Symposium on Artificial Intelligence and Signal Processing (AISP)

View full text Add to dashboard Cite

show abstract

“…All of the traditional algorithms, such as Optimal Brain Damage (OBD) [16], Optimal Brain Surgeon (OBS) [17], and Magnitude-based pruning (MAG) [18], Skeletonization (SKEL) [6], non-contributing units (NC) [7] and Extended Fourier Amplitude Sensitivity Test (EFAST) [13], are available in SNNS (CSS1 is name of algorithm that uses SMV for sparse representation and CSS2 is another technique that uses MMV for sparse representation). Table 1 and 2 demonstrate the results of the simulations.…”

Section: Resultsmentioning

confidence: 99%

Sound Source Localization with CS Based Compressed Neural Network

Dehkordi¹

2012

AJSP

View full text Add to dashboard Cite

Microphone arrays are today employed to specify the sound source locations in numerous real time applications such as speech processing in large rooms or acoustic echo cancellation. Signal sources may exist in the near field or far field with respect to the microphones. Current Neural Networks (NNs) based source localization approaches assume far field narrowband sources. One of the important limitations of these NN-based approaches is making balance between computational complexity and the development of NNs; an architecture that is too large or too small will affect the performance in terms of generalization and computational cost. In the previous analysis, saliency subject has been employed to determine the most suitable structure, however, it is time-consuming and the performance is not robust. In this paper, a family of new algorithms for compression of NNs is presented based on Compressive Sampling (CS) theory. The proposed framework makes it possible to find a sparse structure for NNs, and then the designed neural network is compressed by using CS. The key difference between our algorithm and the state-of-the-art techniques is that the mapping is continuously done using the most effective features; therefore, the proposed method has a fast convergence. The empirical work demonstrates that the proposed algorithm is an effective alternative to traditional methods in terms of accuracy and computational complexity.

show abstract

“…Roughly, the methods for pruning can be classified into two categories: weight pruning and hidden neuron pruning. Examples of weight pruning algorithms include Optimal Brain Damage (OBD) [2], Optimal Brain Surgeon (OBS) [3], and Magnitude-based pruning (MAG) [5]. On the other hand, Skeletonization (SKEL) [6], non-contributing units (NC) [7] and Extended Fourier Amplitude Sensitivity Test (EFAST) [14] all implement hidden neuron pruning.…”

Section: A Traditional Algorithms For Pruning Nnsmentioning

confidence: 99%

A Neural Network pruning approach based on Compressive Sampling

Yang

Bouzerdoum

Phung

2009

2009 International Joint Conference on Neural Networks

View full text Add to dashboard Cite

The balance between computational complexity and the architecture bottlenecks the development of Neural Networks (NNs), An architecture that is too large or too small will influence the performance to a large extent in terms of generalization and computational cost. In the past, saliency analysis has been employed to determine the most suitable structure, however, it is time-consuming and the performance is not robust. In this paper, a family of new algorithms for pruning elements (weighs and hidden neurons) in Neural Networks is presented based on Compressive Sampling (CS) theory. The proposed framework makes it possible to locate the significant elements, and hence find a sparse structure, without computing their saliency. Experiment results are presented which demonstrate the effectiveness of the proposed approach. Abstract-The balance between computational complexity and the architecture bottlenecks the development of Neural Networks (NNs), An architecture that is too large or too small will influence the performance to a large extent in terms of generalization and computational cost. In the past, saliency analysis has been employed to determine the most suitable structure, however, it is time-consuming and the performance is not robust. In this paper, a family of new algorithms for pruning elements (weighs and hidden neurons) in Neural Networks is presented based on Compressive Sampling (CS) theory. The proposed framework makes it possible to locate the significant elements, and hence find a sparse structure, without computing their saliency. Experiment results are presented which demonstrate the effectiveness of the proposed approach.

show abstract

Removal of hidden units and weights for back propagation networks

Cited by 45 publications

References 11 publications

A compressive sensing based compressed neural network for sound source localization

A compressive sensing based compressed neural network for sound source localization

Sound Source Localization with CS Based Compressed Neural Network

A Neural Network pruning approach based on Compressive Sampling

Contact Info

Product

Resources

About