Compressing Convolutional Neural Networks by Pruning Density Peak Filters

Abstract

With the recent development of GPUs, the depth of convolutional neural networks (CNNs) has increased, and its structure has become complex. Hence, it is challenging to deploy them into a hardware device owing to its immense computational cost and memory for storage parameters. We propose a method of pruning a filter located near the density peak, which grasps the density of the filter space for each layer to overcome this problem. The density is calculated in the filter space based on the number of neighboring filters within a certain distance around the filter and the distance to a denser space. Moreover, we do not remove all filters at once, but use a method of pruning a certain number iteratively, so that filters can be evenly pruned in multiple locations with high density inside the filter space. After that, we fine-tune the pruned network to restore their performance. The experimental results show the effectiveness of the proposed method with respect to the other methods using CIFAR-10, and ImageNet dataset on VGGNet and ResNet architecture. Notably, on CIFAR-10, our method reduces 60.8% of FLOPs on ResNet56 with 0.31% validation accuracy improvement. Moreover, we achieve up to 51.9% FLOPs reduction with a little accuracy drop on ImageNet for ResNet34. INDEX TERMSConvolutional neural networks, compressing CNNs, filter pruning, density peak. YUNSEOK JANG received the B.S. degree in electrical and electronic engineering from Yonsei University, Seoul, South Korea, in 2015, where he is currently pursuing the Ph.D. degree with the IT-SOC Laboratory. His research interests include network compression and light object detection. SANGYOUN LEE (Member, IEEE) received the B.S. and M.

COVLIAS 2.0-cXAI: Cloud-Based Explainable Deep Learning System for COVID-19 Lesion Localization in Computed Tomography Scans

“…Choi et al [ 48 ] used SHAP to demonstrate the high-risk factors responsible for higher phosphate. Further, to improve the speed of the AI model, model optimization techniques such as weight clustering and AI model pruning [ 160 , 161 , 162 , 163 , 164 ] can be applied [ 115 , 165 , 166 , 167 , 168 , 169 ]. Techniques such as storage reduction are necessary when dealing with AI solutions [ 51 , 54 , 170 , 171 , 172 ].…”

UNet Deep Learning Architecture for Segmentation of Vascular and Non-Vascular Images: A Microscopic Look at UNet Components Buffered With Pruning, Explainable Artificial Intelligence, and Bias

“…The current pruning literature has been classified into three categories: (i) channel pruning (socalled filter pruning), (ii) network pruning, and (iii) hybrid pruning. The main principle of channel pruning is to cut down the filters at an early stage of the AI model design [218][219][220][221][222][223][224][225][226][227]. We also call it as early pruning.…”