Iterative clustering pruning for convolutional neural networks

Chang, Jingfei; Xue, Ping; Xu, Yiqun; Wei, Zhen

doi:10.1016/j.knosys.2023.110386

Cited by 20 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The efficacy of our proposed algorithm is demonstrated through a representative example in Subfigure 2a. The algorithm detects the saliency of filters, yielding a sequence of indices such as [5,14,7,13,4,1,15,6,10,0,3,8,9,2,12,11]. This sequence effectively identifies the most (index 0, darkest-blue) and least (index 9, brightest-yellow) important filters.…”

Section: Filters Selectionmentioning

confidence: 99%

“…Concerning pruning strategy, there exist two main approaches: one-shot pruning [38,63,79] and multi-shot pruning [4,14]. With the first one, the entire network is pruned in a single shot, and the filters or weights in each layer are either preserved or pruned at once.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, multi-shot pruning is a pruning strategy in which the network is iteratively pruned, and each iteration involves pruning a portion of the redundant filters or weights. As discussed in [4,14], this approach allows the network to adapt to the pruning process, preserving the most important elements and improving the overall accuracy. This method allows for more flexibility in choosing which filters to prune, which results in more effective pruning.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient tensor decomposition-based filter pruning

Pham,

Zniyed,

Nguyen

2024

Neural Networks

View full text Add to dashboard Cite

Section: Filters Selectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient tensor decomposition-based filter pruning

Pham,

Zniyed,

Nguyen

2024

Neural Networks

View full text Add to dashboard Cite

“…Pruning techniques were initially employed in decision tree models and later extended to neural network models [25]. Trough appropriate pruning, the computational structures of neural networks are streamlined, enabling them to eliminate the infuence of redundant parameters and enhance their generalization performance [26,27]. However, due to current limitations in neural network computation methods, pruning methods require trained models.…”

Section: Related Workmentioning

confidence: 99%

Semi-White-Box Strategy: Enhancing Data Efficiency and Interpretability of Convolutional Neural Networks in Image Processing

Wang,

Zeng,

Qin

et al. 2023

International Journal of Intelligent Systems

View full text Add to dashboard Cite

Data-hunger is a persistent challenge in machine learning, particularly in the field of image processing based on convolutional neural networks (CNNs). This study systematically investigates the factors contributing to data-hunger in machine-learning-based image-processing algorithms. The results revealed that the proliferation of model parameters, the lack of interpretability, and the complexity of model structure are significant factors influencing data-hunger. Based on these findings, this paper introduces a novel semi-white-box neural network model construction strategy. This approach effectively reduces the number of model parameters while enhancing the interpretability of model components. It accomplishes this by constraining uninterpretable processes within the model and leveraging prior knowledge of image processing for model. Rather than relying on a single all-in-one model, a semi-white-box model is composed of multiple smaller models, each responsible for extracting fundamental semantic features. The final output is derived from these features and prior knowledge. The proposed strategy holds the potential to substantially decrease data requirements under specific data source conditions while improving the interpretability of model components. Validation experiments are conducted on well-established datasets, including MNIST, Fashion MNIST, CIFAR, and generated data. The results demonstrate the superiority of the semi-white-box strategy over the traditional all-in-one approach in terms of accuracy when trained with equivalent data volumes. Impressively, on the tested datasets, a simplified semi-white-box model achieves performance close to that of ResNet while utilizing a small number of parameters. Furthermore, the semi-white-box strategy offers improved interpretability and parameter reusability features that are challenging to achieve with the all-in-one approach. In conclusion, this paper contributes to mitigating data-hunger challenges in machine-learning-based image processing through the introduction of a novel semi-white-box model construction strategy, backed by empirical evidence of its effectiveness.

show abstract

“…Its widespread adoption In fact, the computational burden of the CNN primarily lies in the convolutional and fully connected layers, as the number of parameters and computational complexity in these two layers determine the model's computation and convergence speed. To address these challenges, various model compression and acceleration techniques have been proposed, such as model pruning [15], model quantization [16], and low-rank decomposition [17], taking into account both model performance and parameter reduction. However, these techniques for large networks are often complex and challenging to implement.…”

Section: Introductionmentioning

confidence: 99%

Detection of balling levels on the surface of SLM formed parts based on finite depth separable convolution network

Qiu,

Jiang,

Lin

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

To meet the real-time requirements of balling levels detection in selective laser melting processes, a modified detection model, called Finite Depth Separable Convolution Network (F-DSCNet), is proposed by optimizing the existing benchmark model (BM) with two lightweight structures: Depth Separable Convolution (DSC) and Global Average Pooling (GAP). This model balances the effect of reducing model’s parameters and increasing model’s structural complexity brought by DSC on the computation and convergence speed of the model, and only introduces DSC in the higher-level convolution layers of the BM. In addition, the GAP structure is adopted instead of the fully connected layer to further reduce the number of parameters and accelerate model training and convergence. The experimental results show that the F-DSCNet model not only maintains high recognition accuracy but also significantly improves the model’s computation and convergence speed, as well as the recognition response time of a single image, exhibiting strong practicality for engineering applications.

show abstract

Iterative clustering pruning for convolutional neural networks

Cited by 20 publications

References 10 publications

Efficient tensor decomposition-based filter pruning

Efficient tensor decomposition-based filter pruning

Semi-White-Box Strategy: Enhancing Data Efficiency and Interpretability of Convolutional Neural Networks in Image Processing

Detection of balling levels on the surface of SLM formed parts based on finite depth separable convolution network

Contact Info

Product

Resources

About