Image and Video Coding Techniques for Ultra-low Latency

Žádník, Jakub; Mäkitalo, Markku; Vanne, Jarno; Jääskeläinen, Pekka

doi:10.1145/3512342

Cited by 12 publications

(2 citation statements)

References 134 publications

(187 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case of recent AAA games where the total texture sizes can range from tens to hundreds of gigabytes, insufficient encoding speed can become a bottleneck during software development. Furthermore, if a texture compression format is used for low‐latency video coding, the encoding process must be completed in real‐time as indicated by Žádník, et al [vMVJ22].…”

Section: Introductionmentioning

confidence: 99%

H‐ETC2: Design of a CPU‐GPU Hybrid ETC2 Encoder

Lee,

Nah

2023

Computer Graphics Forum

View full text Add to dashboard Cite

This paper proposes a novel CPU‐GPU hybrid encoding method based on the ETC2 format, commonly used on mobile platforms. Traditional texture compression techniques often face a trade‐off between encoding speed and quality. For a better trade‐off, our approach utilizes both the CPU and GPU. In a pipeline we designed, the CPU encoder identifies problematic pixel blocks during the encoding process, and the GPU encoder re‐encodes them. Additionally, we carefully improve the base CPU and GPU encoders regarding encoding speed and quality. As a result, our encoder minimizes compression artifacts, increases encoding speed, or achieves both of these goals compared to previous high‐quality offline ETC2 encoders.

show abstract

Section: Introductionmentioning

confidence: 99%

H‐ETC2: Design of a CPU‐GPU Hybrid ETC2 Encoder

Lee,

Nah

2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Several types of low-complexity codecs exist. Real-time texture compression can reach very high encoding speeds compared to other methods at the expense of rather low coding efficiency [1]- [3]. A "mezzanine compression" family of codecs is designed specifically to meet ultra-low latency requirements, with JPEG XS [4] as the newest standard in this family.…”

Section: Introductionmentioning

confidence: 99%

Pruned Lightweight Encoders for Computer Vision

Žádník

Mäkitalo

Jääskeläinen

2022

2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

Latency-critical computer vision systems, such as autonomous driving or drone control, require fast image or video compression when offloading neural network inference to a remote computer. To ensure low latency on a near-sensor edge device, we propose the use of lightweight encoders with constant bitrate and pruned encoding configurations, namely, ASTC and JPEG XS. Pruning introduces significant distortion which we show can be recovered by retraining the neural network with compressed data after decompression. Such an approach does not modify the network architecture or require coding format modifications. By retraining with compressed datasets, we reduced the classification accuracy and segmentation mean intersection over union (mIoU) degradation due to ASTC compression to 4.9-5.0 percentage points (pp) and 4.4-4.0 pp, respectively. With the same method, the mIoU lost due to JPEG XS compression at the main profile was restored to 2.7-2.3 pp. In terms of encoding speed, our ASTC encoder implementation is 2.3x faster than JPEG. Even though the JPEG XS reference encoder requires optimizations to reach low latency, we showed that disabling significance flag coding saves 22-23% of encoding time at the cost of 0.4-0.3 mIoU after retraining.

show abstract