DeepIoT

Abstract

Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing speci c types of neural networks, DeepIoT presents a uni ed approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by nding the minimum number of non-redundant hidden elements, such as lters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. e compressed model generated by DeepIoT can directly use existing deep learning libraries that run on embedded and mobile systems without further modi cations. We conduct experiments with ve di erent sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a signi cant margin. It reduces the size of deep neural networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to 94.5%, and decrease energy consumption by 72.2% to 95.7%. ese improvements are achieved without loss of accuracy. e results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.

PANCODE: Multilevel Partitioning of Neural Networks for Constrained Internet-of-Things Devices

“…Some frameworks for executing neural networks distributed into several IoT devices have been proposed [13], [15], [27], [28], [29], [30]. Sze et al [22] reviewed methods for efficiently executing DNNs, focusing on the inference phase, hardware platforms, and architecture for supporting DNNs.…”

“…DeepIoT [15] compresses CNNs, fully connected neural networks, and Recurrent Neural Networks by extracting redundant neurons. This compression can significantly reduce the DNN size, execution time, and energy consumption without loss of accuracy.…”

“…Existing frameworks that distribute a CNN on IoT devices limit the partitioning into layers [13], [15], [16]. However, this approach might lead to suboptimal results or memoryinvalid partitionings, in which at least one partition needs more memory than the devices provide [14], [17].…”

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

“…Its downside, common to many DL compression techniques, is a permanent decrease in inference accuracy (of ≈ 5%). On the pruning front, PatDNN enables real-time inference using large-scale DL models (e.g., VGG-16, ResNet-50) on mobile devices by harnessing pattern-based model pruning [33], while DeepIoT [48] uses reinforcement learning to guide the pruning process. Both solutions lead to significant model size reductions (90% to 98.9% in case of DeepIoT) and speedups (up to 44.5× in case of PatDNN) with no inference accuracy degradation in certain settings, demonstrating vast opportunities for mobile DL optimisation.…”

Wisdom

“…Firstly, it has become common for distributed applications to be configured for use on mobile [26,30,34] or embedded [10] devices with on the order of GBs of memory and storage, but they are both very energyhungry and expensive at the scale of our application. Another approach is to reduce the required memory and computation by simplifying the models used, for instance in deep neural nets [24,42,43]. However, in cases like ours where mobile phones are too expensive and power hungry and the problem can't be adequately simplified, the use of a co-processor comes to mind.…”