FALF ConvNets: Fatuous auxiliary loss based filter-pruning for efficient deep CNNs

Singh, Pravendra; Kadi, Vinay Sameer Raja; Namboodiri, Vinay P.

doi:10.1016/j.imavis.2019.103857

Cited by 21 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[42] proposed a global filter pruning algorithm, which transforms a vanilla CNN module by multiplying its output by the channel-wise scaling factors. [43] added a fatuous auxiliary task for filter pruning. [44] analyzed most of existing pruning methods mainly focused on classification and few of them conducted systematic research on object detection.…”

Section: Ralated Workmentioning

confidence: 99%

UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration

Chang,

Lu,

Xue

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep convolutional neural networks currently show the most advanced results in many artificial intelligence fields. However, with the continuous increase in the depth and width of CNNs, the number of parameters and floating-point operations (FLOPs) have also increased dramatically. To apply deep CNN to mobile terminals and portable devices, many scholars have recently worked on the compressing and accelerating deep CNN. Based on this, we propose a novel uniform channel pruning (UCP) method and the modified squeeze-and-excitation blocks (MSEB) is used to measure the importance of the channels in the convolutional layers. The unimportant channels, including convolutional kernels related to them, are pruned directly, which greatly reduces the storage cost and number of calculations. There are two types of residual blocks in ResNet. For ResNet with bottlenecks, we use the pruning method with traditional CNN to prune the 3×3 convolutional layer in the middle of the blocks. For ResNet with basic blocks, we propose an approach to consistently prune all residual blocks in the same stage to ensure that the compact network structure is dimensionally correct. Considering that the network loses considerable information after pruning and that the larger the pruning amplitude is, the more information will be lost, we do not choose fine-tuning but retrain from scratch to restore the accuracy of the network after pruning. Finally, we verify our method on CIFAR-10, CIFAR-100 and ILSVRC-2012 for image classification. The results indicate that the performance of the compact network after retraining from scratch, when the pruning rate is small, is better than the original network. Even when the pruning amplitude is large, the accuracy can be maintained or decreased slightly. On the CIFAR-100, when reducing the parameters and FLOPs up to 82% and 62% respectively, the accuracy of VGG-19 even improved by 0.54% after retraining.

show abstract

Section: Ralated Workmentioning

confidence: 99%

UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration

Chang,

Lu,

Xue

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…“A hybrid architecture is used for localized agricultural information dissemination, which is the client-server architecture, in conjunction with mobile applications on smartphones, which can be used to deliver precise agricultural information to the farmers (Chen et al, 2012; Jain et al, 2014). ” Besides the agricultural field, the application of IoT and deep learning is utilized in the diverse area (Lin, 2020; Peixoto et al, 2020; Singh et al, 2020; Yang et al, 2020). This research aims to introduce the latest technology in the paddy field and better crop production by collecting real-time crop status and informing the farmers about it.…”

Section: Introductionmentioning

confidence: 99%

Smart paddy field monitoring system using deep learning and IoT

Sethy

Behera

Kannan

et al. 2021

Concurrent Engineering

View full text Add to dashboard Cite

Paddy is an essential nutrient worldwide. Rice gives 21% of worldwide human per capita energy and 15% of per capita protein. Asia represented 60% of the worldwide populace, about 92% of the world’s rice creation, and 90% of worldwide rice utilization. With the increase in population, the demand for rice is increased. So, the productivity of farming is needed to be enhanced by introducing new technology. Deep learning and IoT are hot topics for research in various fields. This paper suggested a setup comprising deep learning and IoT for monitoring of paddy field remotely. The vgg16 pre-trained network is considered for the identification of paddy leaf diseases and nitrogen status estimation. Here, two strategies are carried out to identify images: transfer learning and deep feature extraction. The deep feature extraction approach is combined with a support vector machine (SVM) to classify images. The transfer learning approach of vgg16 for identifying four types of leaf diseases and prediction of nitrogen status results in 79.86% and 84.88% accuracy. Again, the deep features of Vgg16 and SVM results for identifying four types of leaf diseases and prediction of nitrogen status have achieved an accuracy of 97.31% and 99.02%, respectively. Besides, a framework is suggested for monitoring of paddy field remotely based on IoT and deep learning. The suggested prototype’s superiority is that it controls temperature and humidity like the state-of-the-art and can monitor the additional two aspects, such as detecting nitrogen status and diseases.

show abstract

“…Deep neural networks (DNNs) are state-of-the-art models, responsible for transforming research in the area of vision, language and speech [14,7,4]. Various works [30,31,26,29,27,28,25,20,32] have been proposed for efficient deep learning. These deep network at core performs a linear transformation followed by a non-linear operation using an activation function.…”

Section: Introductionmentioning

confidence: 99%

Cooperative Initialization based Deep Neural Network Training

Singh¹,

Varshney²,

Namboodiri³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Researchers have proposed various activation functions. These activation functions help the deep network to learn non-linear behavior with a significant effect on training dynamics and task performance. The performance of these activations also depends on the initial state of the weight parameters, i.e., different initial state leads to a difference in the performance of a network. In this paper, we have proposed a cooperative initialization for training the deep network using ReLU activation function to improve the network performance. Our approach uses multiple activation functions in the initial few epochs for the update of all sets of weight parameters while training the network. These activation functions cooperate to overcome their drawbacks in the update of weight parameters, which in effect learn better "feature representation" and boost the network performance later. Cooperative initialization based training also helps in reducing the overfitting problem and does not increase the number of parameters, inference (test) time in the final model while improving the performance. Experiments show that our approach outperforms various baselines and, at the same time, performs well over various tasks such as classification and detection. The Top-1 classification accuracy of the model trained using our approach improves by 2.8% for VGG-16 and 2.1% for ResNet-56 on CIFAR-100 dataset.

show abstract

FALF ConvNets: Fatuous auxiliary loss based filter-pruning for efficient deep CNNs

Cited by 21 publications

References 32 publications

UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration

UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration

Smart paddy field monitoring system using deep learning and IoT

Cooperative Initialization based Deep Neural Network Training

Contact Info

Product

Resources

About