Decoding prefix codes

Liddell, M.; Moffat, Alistair

doi:10.1002/spe.741

Cited by 11 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an example, alphabets with more than 300.000 symbols are cited in , for which up to 32 levels would be needed. Whereas we do not have access to those data sets, some tests have been carried out assuming a Gaussian distribution of probabilities for 300‐K symbols.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A fast algorithm for constructing nearly optimal prefix codes

Osorio

González

2015

Softw Pract Exp

View full text Add to dashboard Cite

SUMMARYHuffman algorithm allows for constructing optimal prefix-codes with O.n logn/ complexity. As the number of symbols n grows, so does the complexity of building the code-words. In this paper, a new algorithm and implementation are proposed that achieve nearly optimal coding without sorting the probabilities or building a tree of codes. The complexity is proportional to the maximum code length, making the algorithm especially attractive for large alphabets. The focus is put on achieving almost optimal coding with a fast implementation, suitable for real-time compression of large volumes of data. A practical case example about checkpoint files compression is presented, providing encouraging results.

show abstract

Section: Resultsmentioning

confidence: 99%

“…In , the test data sets proposed in the context of the Text Retrieval Conference (TREC) are presented. In , alphabets from those sets, with more than 300,000 unique symbols, are considered from the point of view of the decoder.…”

Section: Related Workmentioning

confidence: 99%

A fast algorithm for constructing nearly optimal prefix codes

Osorio

González

2015

Softw Pract Exp

View full text Add to dashboard Cite

show abstract

“…Liddell and Moffat surveyed and evaluated techniques to speed up decoding of prefix codes . They report on a multiple‐codeword and multilevel lookup table.…”

Section: Related Workmentioning

confidence: 99%

“…The height of this conceptually infinite tree is capped by the bit-width of our lookup window. Because the resulting tree is not a canonical Huffman tree but a full binary tree, none of the above techniques apply.Liddell and Moffat surveyed and evaluated techniques to speed up decoding of prefix codes 27. They report on a multiple-codeword and multilevel lookuptable.…”

mentioning

confidence: 99%

JParEnt: Parallel entropy decoding for JPEG decompression on heterogeneous multicore architectures

Sodsong

Jung

Park

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary The JPEG format employs Huffman codes to compress the entropy data of an image. Huffman codewords are of variable length, which makes parallel entropy decoding a difficult problem. To determine the start position of a codeword in the bitstream, the previous codeword must be decoded first. We present JParEnt, a new approach to parallel entropy decoding for JPEG decompression on heterogeneous multicores. JParEnt conducts JPEG decompression in two steps: (1) an efficient sequential scan of the entropy data on the CPU to determine the start‐positions (boundaries) of coefficient blocks in the bitstream, followed by (2) a parallel entropy decoding step on the graphics processing unit (GPU). The block boundary scan constitutes a reinterpretation of the Huffman‐coded entropy data to determine codeword boundaries in the bitstream. We introduce a dynamic workload partitioning scheme to account for GPUs of low compute power relative to the CPU. This configuration has become common with the advent of SoCs with integrated graphics processors (IGPs). We leverage additional parallelism through pipelined execution across CPU and GPU. For systems providing a unified address space between CPU and GPU, we employ zero‐copy to completely eliminate the data transfer overhead. Our experimental evaluation of JParEnt was conducted on six heterogeneous multicore systems: one server and two desktops with dedicated GPUs, one desktop with an IGP, and two embedded systems. For a selection of more than 1000 JPEG images, JParEnt outperforms the SIMD–implementation of the libjpeg‐turbo library by up to a factor of 4.3×, and the previously fastest JPEG decompression method for heterogeneous multicores by up to a factor of 2.2×. JParEnt's entropy data scan consumes 45% of the entropy decoding time of libjpeg‐turbo on average. Given this new ratio for the sequential part of JPEG decompression, JParEnt achieves up to 97% of the maximum attainable speedup (95% on average). On the IGP‐based desktop platform, JParEnt achieves energy savings of up to 45% compared to libjpeg‐turbo's SIMD‐implementation.

show abstract

“…Having ruled out conventional entropy coders, we must consider alternatives. Variable‐length codes are generally inefficient to decode because of their bit‐level accesses, and even table‐based approaches are not much faster . We therefore turn to fixed‐length codes.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Lossless asymmetric single instruction multiple data codec

Wassenberg

2011

Softw Pract Exp

View full text Add to dashboard Cite

This report introduces a new lossless asymmetric single instruction multiple data codec designed for extremely efficient decompression of large satellite images. A throughput in excess of 3GB/s allows decompression to proceed in parallel with asynchronous transfers from fast block devices such as disk arrays. This is made possible by a simple and fast single instruction multiple data entropy coder that removes leading null bits. Our main contribution is a new approach for vectorized prediction and encoding. Unlike previous approaches that treat the entropy coder as a black box, we account for its properties in the design of the predictor. The resulting compressed stream is 1.2 to 1.5 times as large as JPEG-2000, but can be decompressed 100 times as quickly - even faster than copying uncompressed data in memory. Applications include streaming decompression for out of core visualization. To the best of our knowledge, this is the first entirely vectorized algorithm for lossless compression

show abstract

Decoding prefix codes

Cited by 11 publications

References 23 publications

A fast algorithm for constructing nearly optimal prefix codes

A fast algorithm for constructing nearly optimal prefix codes

JParEnt: Parallel entropy decoding for JPEG decompression on heterogeneous multicore architectures

Lossless asymmetric single instruction multiple data codec

Contact Info

Product

Resources

About