EACP: An effective automatic channel pruning for neural networks

Liu, Yajun; Wu, Dakui; Zhou, Wenju; Fan, Kefeng; Zhou, Zhiheng

doi:10.1016/j.neucom.2023.01.014

Cited by 16 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 shows pruning results of GoogLeNet on CIFAR-10. In all conducted cases, CORING consistently outperforms other methods [35,37,46] in every way. Therefore, inception modules can use our method to achieve high performance in comparison to cutting-edge techniques.…”

Section: Results and Analysismentioning

confidence: 81%

See 1 more Smart Citation

Efficient tensor decomposition-based filter pruning

Pham,

Zniyed,

Nguyen

2024

Neural Networks

View full text Add to dashboard Cite

Section: Results and Analysismentioning

confidence: 81%

“…Comparison. CORING is compared with 44 SOTAs in the fields of structured pruning [1,3,5,6,7,11,12,15,16,18,19,23,26,29,30,31,32,34,35,36,37,38,39,40,41,43,45,46,47,48,49,56,58,63,64,66,68,73,76,78,79,80,83,85]. For a fair comparison, all available baseline models are identical.…”

Section: Experimental Settingsmentioning

confidence: 99%

Efficient tensor decomposition-based filter pruning

Pham,

Zniyed,

Nguyen

2024

Neural Networks

View full text Add to dashboard Cite

“…where 𝐼 denotes the identity matrix. Then the Hutchinson algorithm [4] can be utilized to compute the Hessian trace: 6), (7), and (8), respectively.…”

Section: Rank Selection Based On Sensitivitymentioning

confidence: 99%

“…Several popular techniques to decrease redundancy in neural network parameters include pruning [3,4], sparsification [5,6], quantization [7,8], and low-rank tensor decomposition [9,10]. Among these techniques, the tensor decomposition-based compression method is currently attracting growing interest.…”

Section: Introductionmentioning

confidence: 99%

Stable Low-Rank CP Decomposition for Compression of Convolutional Neural Networks Based on Sensitivity

Yang,

Liu

2024

Applied Sciences

View full text Add to dashboard Cite

Modern convolutional neural networks (CNNs) play a crucial role in computer vision applications. The intricacy of the application scenarios and the growing dataset both significantly raise the complexity of CNNs. As a result, they are often overparameterized and have significant computational costs. One potential solution for optimizing and compressing the CNNs is to replace convolutional layers with low-rank tensor decomposition. The most suitable technique for this is Canonical Polyadic (CP) decomposition. However, there are two primary issues with CP decomposition that lead to a significant loss in accuracy. Firstly, the selection of tensor ranks for CP decomposition is an unsolved issue. Secondly, degeneracy and instability are common problems in the CP decomposition of contractional tensors, which makes fine-tuning the compressed model difficult. In this study, a novel approach was proposed for compressing CNNs by using CP decomposition. The first step involves using the sensitivity of convolutional layers to determine the tensor ranks for CP decomposition effectively. Subsequently, to address the degeneracy issue and enhance the stability of the CP decomposition, two novel techniques were incorporated: optimization with sensitivity constraints and iterative fine-tuning based on sensitivity order. Finally, the proposed method was examined on common CNN structures for image classification tasks and demonstrated that it provides stable performance and significantly fewer reductions in classification accuracy.

show abstract

“…To address these challenges, researchers and engineers have been exploring various methods for DNNs compression, including pruning (Tmamna et al, 2021; Liu et al, 2023), quantization (Bablani et al, 2023; Ma et al, 2023; Tmamna et al, 2023), and knowledge distillation (Yim et al, 2017; Zhao et al, 2020). Pruning involves removing unnecessary connections or parameters in the network, resulting in a smaller model size.…”

Section: Introductionmentioning

confidence: 99%

Bare‐Bones particle Swarm optimization‐based quantization for fast and energy efficient convolutional neural networks

Tmamna,

Ayed,

Fourati

et al. 2023

Expert Systems

View full text Add to dashboard Cite

Neural network quantization is a critical method for reducing memory usage and computational complexity in deep learning models, making them more suitable for deployment on resource‐constrained devices. In this article, we propose a method called BBPSO‐Quantizer, which utilizes an enhanced Bare‐Bones Particle Swarm Optimization algorithm, to address the challenging problem of mixed precision quantization of convolutional neural networks (CNNs). Our proposed algorithm leverages a new population initialization, a robust screening process, and a local search strategy to improve the search performance and guide the population towards a feasible region. Additionally, Deb's constraint handling method is incorporated to ensure that the optimized solutions satisfy the functional constraints. The effectiveness of our BBPSO‐Quantizer is evaluated on various state‐of‐the‐art CNN architectures, including VGG, DenseNet, ResNet, and MobileNetV2, using CIFAR‐10, CIFAR‐100, and Tiny ImageNet datasets. Comparative results demonstrate that our method delivers an excellent tradeoff between accuracy and computational efficiency.

show abstract

EACP: An effective automatic channel pruning for neural networks

Cited by 16 publications

References 31 publications

Efficient tensor decomposition-based filter pruning

Efficient tensor decomposition-based filter pruning

Stable Low-Rank CP Decomposition for Compression of Convolutional Neural Networks Based on Sensitivity

Bare‐Bones particle Swarm optimization‐based quantization for fast and energy efficient convolutional neural networks

Contact Info

Product

Resources

About