Automatic channel pruning via clustering and swarm intelligence optimization for CNN

Chang, Jingfei; Xue, Ping; Xu, Yiqun; Wei, Zhen

doi:10.1007/s10489-022-03508-1

Cited by 15 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this subsection, we compare the methods on CIFAR-10 and CIFAR-100. Among the compared methods, KPGP [42], SFP [51], FPGM [49] and ACP [50] are the state-of-the-art channel pruning methods. The results of these competing methods have been reported in the original articles.…”

Section: Results On Cifar-100mentioning

confidence: 99%

See 1 more Smart Citation

Automatic Group-Based Structured Pruning for Deep Convolutional Networks

Wei

Wang

Hua³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Structured pruning methods have been used in several convolutional neural networks (CNNs). However, group-based structured pruning is a challenging task. In previous methods, the number of groups is manually determined for all layers, which is suboptimal. Moreover, which kernels should be appropriately removed? Model accuracy may be significantly reduced when the number of kernels is removed. To address these challenges, we propose an automatic group-based structured pruning method with reinforcement learning, named AGSPRL, which can generate pruned models with different compression rates automatically. We first develop a reinforcement learning (RL) framework to learn the pruning rate for group-based channel pruning layer by layer. Then, based on the learned kernel pruning rate, we propose an efficient group configuration algorithm to adaptively determine the number of groups for each convolution layer. Finally, we introduce a channel pruning method with an attention mechanism as a tiny auxiliary filter selector for each group to dynamically determine which part of the kernels should be selected into the group convolution and which part of the kernels should be removed. To demonstrate the efficiency of our method, we apply it to a variety of CNNs in classification and detection datasets. The experimental results show that the AGSPRL not only adaptively but also accurately configures the number of groups. The accuracy is reduced by less than 1% and improved by 1%. Moreover, compared to other state-of-the-art methods, AGSPRL is more effective and has less accuracy loss.

show abstract

Section: Results On Cifar-100mentioning

confidence: 99%

“…A common data augmentation scheme including random cropping and mirroring [4], [49] is adopted [10]. After pruning, the pruned model is retrained for [50]. The performance is evaluated by mean average precision (mAP).…”

Section: A Datasets and Settingmentioning

confidence: 99%

Automatic Group-Based Structured Pruning for Deep Convolutional Networks

Wei

Wang

Hua³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In [66], [67], and [68], different types of convolutional neural networks have been optimized using swarm intelligence algorithms. In [66], three types of convolutional neural networks have been used, namely, the Visual Geometry Group Neural Network (VGGNet), the Residual Neural Network (ResNet), and the GoogLeNet. The VGGNet is a convolutional neural network with either 16 or 19 layers.…”

Section: Optimizing Different Types Of Artificial Neural Networkmentioning

confidence: 99%

“…In [70], the optimal connection weights and biases of a convolutional neural network are determined by a swarm intelligence algorithm. In [66], [68], [71], and [72], the structure of convolutional neural networks is optimized using swarm intelligence algorithms.…”

Section: Optimizing Artificial Neural Network Weights Biases and Stru...mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on the Optimization of Artificial Neural Networks Using Swarm Intelligence Algorithms

2023

View full text Add to dashboard Cite

Artificial Neural Networks (ANNs) are becoming increasingly useful in numerous areas as they have a myriad of applications. Prior to using ANNs, the network structure needs to be determined and the ANN needs to be trained. The network structure is usually chosen based on trial and error. The training, which consists of finding the optimal connection weights and biases of the ANN, is usually done using gradient-descent algorithms. It has been found that swarm intelligence algorithms are favorable for both determining the network structure and for the training of ANNs. This is because they are able to determine the network structure in an intelligent way, and they are better at finding the most optimal connection weights and biases during the training as opposed to conventional algorithms. Recently, a number of swarm intelligence algorithms have been employed for optimizing different types of neural networks. However, there is no comprehensive survey on the swarm intelligence algorithms used for optimizing ANNs. In this paper, we present a review of the different types of ANNs optimized using swarm intelligence algorithms, the way the ANNs are optimized, the different swarm intelligence algorithms used, and the applications of the ANNs optimized by swarm intelligence algorithms.INDEX TERMS Artificial neural network, swarm intelligence, optimization. I. INTRODUCTIONArtificial Neural Networks (ANNs) are computational models that simulate the biological neural network that constitutes the human brain to generate inferences based on certain given information. They are suitable for both supervised and unsupervised learning for solving a myriad of classification, regression, clustering, and association problems in a multitude of areas. Notably, ANN has been a prominent algorithm in the domain of machine learning, and has paved the way for the advancement in multiple areas such as natural language processing, fraud detection, computational biology, computer vision, unassisted control of vehicles, speech recognition, medical diagnosis and recommendation systems [1]. Recently, ANNs have been applied for making decisions in healthcare organizations [2], for forecasting the energy use in buildings [3], for the development of greenhouse technol-The associate editor coordinating the review of this manuscript and approving it for publication was Yeliz Karaca .

show abstract