DeepSIC: Deep Semantic Image Compression

Abstract

Incorporating semantic information into the codecs during image compression can significantly reduce the repetitive computation of fundamental semantic analysis (such as object recognition) in client-side applications. The same practice also enable the compressed code to carry the image semantic information during storage and transmission. In this paper, we propose a concept called Deep Semantic Image Compression (DeepSIC) and put forward two novel architectures that aim to reconstruct the compressed image and generate corresponding semantic representations at the same time. The first architecture performs semantic analysis in the encoding process by reserving a portion of the bits from the compressed code to store the semantic representations. The second performs semantic analysis in the decoding step with the feature maps that are embedded in the compressed code. In both architectures, the feature maps are shared by the compression and the semantic analytics modules. To validate our approaches, we conduct experiments on the publicly available benchmarking datasets and achieve promising results. We also provide a thorough analysis of the advantages and disadvantages of the proposed technique.

DSSLIC: Deep Semantic Segmentation-based Layered Image Compression

“…Recently, there have also been some efforts in combining some computer vision tasks and image compression in one framework. In [9,18], the authors tried to use the feature maps from learning-based image compression to help other tasks such as image classification and semantic segmentation although the results from other tasks were not used to help the compression part. In [2], a segmentation map-based image synthesis model was proposed, which targeted extremely low bit rates (< 0.1 bits/pixel), and used synthesized images for non-important regions.…”

Near-Lossless Deep Feature Compression for Collaborative Intelligence

“…where V ∈ R N ×M ×C is the feature tensor with N rows, M columns, and C channels at the point of split, V is the quantized feature tensor, and min(V) and max(V) are the minimum and maximum value in V, respectively. In the studies performed so far [3], [4], [7], this uniform n-bit quantization was shown to have negligible effect on image classification and object detection accuracy, for n ≥ 6. For this reason, when such uniform quantizer is followed up by a lossless encoder, we refer to the resulting approach as near-lossless compression.…”

“…If the block is at the left (top) boundary, p j * is used as the left (top) value. The two Fil modes are based on 3tap filters with coefficients [3,7,22]/32 or [14, 0, 18]/32 [13], and use the top-left, top, and left feature values to predict the current value. Again, at the boundaries, the unavailable values are replaced by p j * .…”

“…A related work [7] presented semantic image compression by encoding deep features and then reconstructing the input image from them. The compression was based on uniform quantization followed by context-based adaptive arithmetic coding (CABAC) from H.264 [8].…”

Improved hybrid layered image compression using deep learning and traditional codecs

“…The similarity is measured by perceptual loss based on the feature map of a pretrained AlexNet. In [18,2], the feature maps were extracted from learning-based image compression to complete image segmentation or image classification. Based on the work in [11,18,2], the work [2] used segmentation map-based image synthesis to train the discriminator of GAN, which used synthesized images for non-important regions and obtained very good performance at low bite rates (<0.1 bits/pixel).…”