JPEG XS—A New Standard for Visually Lossless Low-Latency Lightweight Image Coding

Descampe, Antonin; Richter, Thomas; Ebrahimi, Touradj; Foessel, Siegfried; Keinert, Joachim; Bruylants, Tim; Pellegrin, Pascal; Buysschaert, Charles; Rouvroy, Gaël

doi:10.1109/jproc.2021.3080916

Cited by 18 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the reference encoder, the wavelet transform is performed over the whole frame before the coding of individual precincts. However, it is possible to interleave the wavelet transform with the precinct coding as the latency of the wavelet transform ranges from a few pixels to 6 lines [4].…”

Section: Discussionmentioning

confidence: 99%

“…The JPEG XS rate allocation can predict the bitrate precisely, unlike HTJ2K where a precise rate allocation would require a significant additional complexity [4]. The latest version of the standard also supports direct Bayer data compression [13] which can be used to bypass the traditional image processing pipeline at the sensor side and thus save latency.…”

Section: B Jpeg Xsmentioning

confidence: 99%

“…Real-time texture compression can reach very high encoding speeds compared to other methods at the expense of rather low coding efficiency [1]- [3]. A "mezzanine compression" family of codecs is designed specifically to meet ultra-low latency requirements, with JPEG XS [4] as the newest standard in this family. The recently standardized high throughput JPEG 2000 (HTJ2K) [5] simplifies the otherwise complex JPEG 2000 [6] with the goal of 10x throughput improvement.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Pruned Lightweight Encoders for Computer Vision

Žádník

Mäkitalo

Jääskeläinen

2022

2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

Latency-critical computer vision systems, such as autonomous driving or drone control, require fast image or video compression when offloading neural network inference to a remote computer. To ensure low latency on a near-sensor edge device, we propose the use of lightweight encoders with constant bitrate and pruned encoding configurations, namely, ASTC and JPEG XS. Pruning introduces significant distortion which we show can be recovered by retraining the neural network with compressed data after decompression. Such an approach does not modify the network architecture or require coding format modifications. By retraining with compressed datasets, we reduced the classification accuracy and segmentation mean intersection over union (mIoU) degradation due to ASTC compression to 4.9-5.0 percentage points (pp) and 4.4-4.0 pp, respectively. With the same method, the mIoU lost due to JPEG XS compression at the main profile was restored to 2.7-2.3 pp. In terms of encoding speed, our ASTC encoder implementation is 2.3x faster than JPEG. Even though the JPEG XS reference encoder requires optimizations to reach low latency, we showed that disabling significance flag coding saves 22-23% of encoding time at the cost of 0.4-0.3 mIoU after retraining.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Jpeg Xsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pruned Lightweight Encoders for Computer Vision

Žádník

Mäkitalo

Jääskeläinen

2022

2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

show abstract

“…A JPEG-XS codestream consists of groups of consecutive lines, named precincts, that are independently encoded. Each precinct is made of Low-Frequency (LF) and High-Frequency (HF) packets, each packet being split into significance, bitplane count, and data subpackets [22,23]. The data subpackets encode the wavelet coefficients associated with the group of lines as a set of bitplanes.…”

Section: Jpeg-xs In Front Of Lossesmentioning

confidence: 99%

Forward Error Correction Applied to JPEG-XS Codestreams

Legrand

Macq

Vleeschouwer

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

JPEG-XS offers low complexity image compression for applications with constrained but reasonable bit-rate, and low latency. Our paper explores the deployment of JPEG-XS on lossy packet networks. To preserve low latency, Forward Error Correction (FEC) is envisioned as the protection mechanism of interest. Although the JPEG-XS codestream is not scalable in essence, we observe that the loss of a codestream fraction impacts the decoded image quality differently, depending on whether this codestream fraction corresponds to codestream headers, to coefficient significance information, or to low/high frequency data. Hence, we propose a rate-distortion optimal unequal error protection scheme that adapts the redundancy level of Reed-Solomon codes according to the rate of channel losses and the type of information protected by the code. Our experiments demonstrate that, at 5% loss rates, it reduces the Mean Squared Error by up to 92% and 65%, compared to a transmission without and with optimal but equal protection, respectively.

show abstract

“…This compression-first approach also allows us to process images more freely than the demosaicing-first one which almost automatically performs various image processings before compression. In light of these facts, although many traditional image-compression methods take the demosaicingfirst approach, as do standards such as JPEG [5] and JPEG 2000 [6], JPEG XS 2nd Edition [7] takes both approaches. We believe that the compression-first approach will get more attention in the future from not only professional photographers and designers but also typical consumers.…”

Section: Introductionmentioning

confidence: 99%

Edge-Aware Extended Star-Tetrix Transforms for CFA-Sampled Raw Camera Image Compression

Suzuki

Huang

2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Codecs using spectral-spatial transforms efficiently compress raw camera images captured with a color filter array (CFA-sampled raw images) by changing their RGB color space into a decorrelated color space. This study describes two types of spectral-spatial transform, called extended Star-Tetrix transforms (XSTTs), and their edge-aware versions, called edge-aware XSTTs (EXSTTs), with no extra bits (side information) and little extra complexity. They are obtained by (i) extending the Star-Tetrix transform (STT), which is one of the latest spectral-spatial transforms, to a new version of our previously proposed waveletbased spectral-spatial transform and a simpler version, (ii) considering that each 2-D predict step of the wavelet transform is a combination of two 1-D diagonal or horizontal-vertical transforms, and (iii) weighting the transforms along the edge directions in the images. Compared with XSTTs, the EXSTTs can decorrelate CFA-sampled raw images well: they reduce the difference in energy between the two green components by about 3.38-30.08 % for high-quality camera images and 8.97-14.47 % for mobile phone images. The experiments on JPEG 2000-based lossless and lossy compression of CFA-sampled raw images show better performance than conventional methods. For high-quality camera images, the XSTTs/EXSTTs produce results equal to or better than the conventional methods: especially for images with many edges, the type-I EXSTT improves them by about 0.03-0.19 bpp in average lossless bitrate and the XSTTs improve them by about 0.16-0.96 dB in average Bjøntegaard delta peak signal-tonoise ratio. For mobile phone images, our previous work perform the best, whereas the XSTTs/EXSTTs show similar trends to the case of high-quality camera images.

show abstract

JPEG XS—A New Standard for Visually Lossless Low-Latency Lightweight Image Coding

Cited by 18 publications

References 8 publications

Pruned Lightweight Encoders for Computer Vision

Pruned Lightweight Encoders for Computer Vision

Forward Error Correction Applied to JPEG-XS Codestreams

Edge-Aware Extended Star-Tetrix Transforms for CFA-Sampled Raw Camera Image Compression

Contact Info

Product

Resources

About