Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices

Abstract

Deep learning has celebrated resounding successes in many application areas of relevance to the Internet of Things (IoT), such as computer vision and machine listening. These technologies must ultimately be brought directly to the edge to fully harness the power of deep leaning for the IoT. The obvious challenge is that deep learning techniques can only be implemented on strictly resource-constrained edge devices if the models are radically downsized. This task relies on different model compression techniques, such as network pruning, quantization, and the recent advancement of XNOR-Net. This study examines the suitability of these techniques for audio classification on microcontrollers. We present an application of XNOR-Net for end-to-end raw audio classification and a comprehensive empirical study comparing this approach with pruning-and-quantization methods. We show that raw audio classification with XNOR yields comparable performance to regular full precision networks for small numbers of classes while reducing memory requirements 32-fold and computation requirements 58-fold. However, as the number of classes increases significantly, performance degrades, and pruning-and-quantization based compression techniques take over as the preferred technique being able to satisfy the same space constraints but requiring approximately 8x more computation. We show that these insights are consistent between raw audio classification and image classification using standard benchmark sets. To the best of our knowledge, this is the first study to apply XNOR to end-to-end audio classification and evaluate it in the context of alternative techniques. All codes are publicly available on GitHub.

Application of Deep Learning to Production Forecasting in Intelligent Agricultural Product Supply Chain

“…In the IoT context, intelligent agricultural supply chain yield prediction based on deep learning plays an important role in reducing the risk of fracturing the agricultural supply chain, further alleviating the vulnerability of the agricultural supply chain and enhancing its resilience. For example, deep reinforcement learning is used to achieve scalable multi-product inventory control with lead time constraints [25].…”

Real-Time Acoustic Perception for Automotive Applications

“…The I-SPOT Project embodies such considerations in the algorithmic design, via the introduction of a co-design workflow to promote the joint optimization of model accuracy and complexity. This new research trend is still weakly explored in the audio signal processing research community [22], [23] as compared to the well-established research on network quantization and pruning for image processing using deep learning techniques [24]. This requires strengthening the interconnection between the heterogeneous algorithm development exploiting deep learning in combination with traditional signal processing on one hand, and versatile reconfigurable hardware architectures optimized for the automotive setting on the other hand.…”

An Intelligent Low-Complexity Computing Interleaving Wavelet Scattering Based Mobile Shuffling Network for Acoustic Scene Classification

“…Our differentiation lies with the inclusion of shuffling modules, which has improved the generalization of our model. Lastly, [48] reuses the CNN model proposed by [46] and performs a series of pruning and quantization. Their best result that pushes the model size to 133.12KB only stands at 70.93%.…”