End-to-end Optimized Image Compression

Ballé, Johannes; Laparra, Valero; Simoncelli, Eero P.

doi:10.48550/arxiv.1611.01704

Cited by 164 publications

(370 citation statements)

References 33 publications

Supporting

Mentioning

367

Contrasting

Order By: Relevance

“…Using our framework, it may therefore become possible to train deep learning models directly on these compressed datasets, which is challenging for traditional compressed formats such as JPEG (although image-specific exceptions such as Nash et al (2021) exist). In addition, learning distributions of functa is likely to improve entropy coding and hence compression for these frameworks (Ballé et al, 2016).…”

Section: Conclusion Limitations and Future Workmentioning

confidence: 99%

From data to functa: Your data point is a function and you can treat it like one

Dupont¹,

Kim²,

Eslami³

et al. 2022

Preprint

View full text Add to dashboard Cite

It is common practice in deep learning to represent a measurement of the world on a discrete grid, e.g. a 2D grid of pixels. However, the underlying signal represented by these measurements is often continuous, e.g. the scene depicted in an image. A powerful continuous alternative is then to represent these measurements using an implicit neural representation, a neural function trained to output the appropriate measurement value for any input spatial location. In this paper, we take this idea to its next level: what would it take to perform deep learning on these functions instead, treating them as data? In this context we refer to the data as functa, and propose a framework for deep learning on functa. This view presents a number of challenges around efficient conversion from data to functa, compact representation of functa, and effectively solving downstream tasks on functa. We outline a recipe to overcome these challenges and apply it to a wide range of data modalities including images, 3D shapes, neural radiance fields (NeRF) and data on manifolds. We demonstrate that this approach has various compelling properties across data modalities, in particular on the canonical tasks of generative modeling, data imputation, novel view synthesis and classification.

show abstract

Section: Conclusion Limitations and Future Workmentioning

confidence: 99%

From data to functa: Your data point is a function and you can treat it like one

Dupont¹,

Kim²,

Eslami³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The structures of generative models to estimate likelihood are diverse. Variational auto-encoder based generative models are first used in learning-based lossy image compression like [2], [3] and [4]. Similar structures are adopted for learning-based lossless image compression like [5].…”

Section: Related Workmentioning

confidence: 99%

End-to-end lossless compression of high precision depth maps guided by pseudo-residual

Wu¹,

Gao²

2022

Preprint

View full text Add to dashboard Cite

As a fundamental data format representing spatial information, depth map is widely used in signal processing and computer vision fields. Massive amount of high precision depth maps are produced with the rapid development of equipment like laser scanner or LiDAR. Therefore, it is urgent to explore a new compression method with better compression ratio for high precision depth maps. Utilizing the wide spread deep learning environment, we propose an end-to-end learning-based lossless compression method for high precision depth maps. The whole process is comprised of two sub-processes, named pre-processing of depth maps and deep lossless compression of processed depth maps. The deep lossless compression network consists of two sub-networks, named lossy compression network and lossless compression network. We leverage the concept of pseudo-residual to guide the generation of distribution for residual and avoid introducing context models. Our end-to-end lossless compression network achieves competitive performance over engineered codecs and has low computational cost.

show abstract

“…CAE is the most popular framework for the LIC. Based on a well-designed network architecture, CAE with factorized prior [26] can reach a comparable coding gain with JPEG2000. With a more powerful entropy model called hyperprior, [27] can achieve the BPG performance.…”

Section: A Learned Image Compressionmentioning

confidence: 99%

End-To-End Learned Image Compression With Fixed Point Weight Quantization

Sun

Cheng

Takeuchi

et al. 2020

2020 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

End-to-end Learned image compression (LIC) has reached the traditional hand-crafted methods such as BPG (HEVC intra) in terms of the coding gain. However, the large network size prohibits the usage of LIC on resource-limited embedded systems. This paper reduces the network complexity by quantizing both weights and activations. 1) For the weight quantization, we study different kinds of grouping and quantization scheme at first. A channel-wise non-linear quantization scheme is determined based on the coding gain analysis. After that, we propose a fine tuning scheme to clip the weights within a certain range so that the quantization error can be reduced. 2) For the activation quantization, we first propose multiple nonlinear quantization codebooks with different maximum dynamic ranges. By selecting an optimal one through a multiplexer, the quantization range can be saturated to the greatest extent. In addition, we also exploit the mean-removed quantization for the analysis transform outputs in order to reduce the bit-width cost for the specific channel with the large non-zero mean. By quantizing each weight and activation element from 32-bit floating point to 8-bit fixed point, the memory cost for both weight and activation can be reduced by 75% with negligible coding performance loss. As a result, our quantized LIC can still outperform BPG in terms of MS-SSIM. To our best knowledge, this is the first work to give a complete analysis on the coding gain and the memory cost for a quantized LIC network, which validates the feasibility of the hardware implementation.

show abstract

End-to-end Optimized Image Compression

Cited by 164 publications

References 33 publications

From data to functa: Your data point is a function and you can treat it like one

From data to functa: Your data point is a function and you can treat it like one

End-to-end lossless compression of high precision depth maps guided by pseudo-residual

End-To-End Learned Image Compression With Fixed Point Weight Quantization

Contact Info

Product

Resources

About