Near-Lossless Deep Feature Compression for Collaborative Intelligence

Abstract

Collaborative intelligence is a new paradigm for efficient deployment of deep neural networks across the mobilecloud infrastructure. By dividing the network between the mobile and the cloud, it is possible to distribute the computational workload such that the overall energy and/or latency of the system is minimized. However, this necessitates sending deep feature data from the mobile to the cloud in order to perform inference. In this work, we examine the differences between the deep feature data and natural image data, and propose a simple and effective near-lossless deep feature compressor. The proposed method achieves up to 5% bit rate reduction compared to HEVC-Intra and even more against other popular image codecs. Finally, we suggest an approach for reconstructing the input image from compressed deep features that could serve to supplement the inference performed by the deep model.

BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services

“…Furthermore, pushing all computations toward the cloud can lead to congestion in a scenario where a large number of mobile devices simultaneously send data to the cloud. As a compromise between the mobile-only and the cloud-only approach, recently, a body of research work has been investigating the idea of splitting a deep inference network between the mobile and cloud [6][7][8][9][10][11][12]. In this approach, which is referred to as collaborative intelligence, the computations associated with initial layers of the inference network are performed on the mobile device, and the feature tensor (activations) of the last computed layer is sent to the cloud for the remainder of computations.…”

“…In research studies investigating collaborative intelligence, a given deep network is split between the mobile device and the cloud without any modification to the network architecture itself [6,[8][9][10][11][12]. In this paper, we investigate altering the underlying deep model architecture to make it collaborative intelligence friendly.…”

Back-And-Forth Prediction for Deep Tensor Compression

“…However, this approach can cause congestion problems due to the growth of the volume of data transmitted over the network and the number of devices linked to the cloud. To address this problem, recent studies in collaborative intelligence have developed optimized deployment strategies [2][3][4][5][6][7].…”

“…Furthermore, considering the impact of the volume of data on the congestion problem, it is desirable to compress and transfer a lesser volume of data to the cloud, unless the inference performance loss is large [3]. Previous studies [4,5] have explored the efficacy of compressing deep feature tensors using conventional standard codecs, in the context of object detection and image classification. Also, [6] suggests a method to first reduce the dimensionality of the deep feature tensor, then compress it using a codec.…”

Multi-Task Learning with Compressible Features for Collaborative Intelligence

“…In CI, the mobile device runs a part of the deep model between the input and some layer, generates a set of deep features, and sends them to the cloud for further processing by the remainder of the deep model, which resides in the cloud. In this context, the issues of deep feature compression [3,4,5] and transmission [6] become important. In [3], This work was supported in part by NSERC Grant RGPIN-2016-04590.…”