Is Huffman coding dead?

Bookstein, Abraham; Klein, Shmuel T.

doi:10.1007/bf02243872

Cited by 42 publications

(32 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventional lossless compression algorithms such as the one used in the GNU gzip program are expensive in terms of both computation and memory access because most of them maintain an entropy-based mapping table and look up the table for compression and decompression [15]. Thus, although they may be useful for disk-resident indexes, they are not adequate for main memory indexes.…”

Section: Overlap Check Without Decompressionmentioning

confidence: 99%

Optimizing multidimensional index trees for main memory access

2001

View full text Add to dashboard Cite

Recent studies have shown that cache-conscious indexes such as the CSB+-tree outperform conventional main memory indexes such as the T-tree. The key idea of these cache-conscious indexes is to eliminate most of child pointers from a node to increase the fanout of the tree. When the node size is chosen in the order of the cache block size, this pointer elimination effectively reduces the tree height, and thus improves the cache behavior of the index. However, the pointer elimination cannot be directly applied to multidimensional index structures such as the R-tree, where the size of a key, typically, an MBR (minimum bounding rectangle), is much larger than that of a pointer. Simple elimination of fourbyte pointers does not help much to pack more entries in a node. This paper proposes a cache-conscious version of the R-tree called the CR-tree. To pack more entries in a node, the CR-tree compresses MBR keys, which occupy almost 80% of index data in the two-dimensional case. It first represents the coordinates of an MBR key relatively to the lower left corner of its parent MBR to eliminate the leading 0's from the relative coordinate representation. Then, it quantizes the relative coordinates with a fixed number of bits to further cut off the trailing less significant bits. Consequently, the CR-tree becomes significantly wider and smaller than the ordinary R-tree. Our experimental and analytical study shows that the two-dimensional CR-tree performs search up to 2.5 times faster than the ordinary R-tree while maintaining similar update performance and consuming about 60% less memory space.

show abstract

Section: Overlap Check Without Decompressionmentioning

confidence: 99%

Optimizing multidimensional index trees for main memory access

2001

View full text Add to dashboard Cite

show abstract

“…Sharing the same code table over many streams can reduce this storage overhead. Huffman encoding generally benefits from large alphabets with an uneven distribution of used characters [21]. It is therefore considered not to be ideal for efficient compression of DNA sequences [22].…”

Section: Statistical Algorithmsmentioning

confidence: 99%

Trends in Genome Compression

Wandelt

Bux

Leser

2014

CBIO

View full text Add to dashboard Cite

Technological advancements in high-throughput sequencing have lead to a tremendous increase in the amount of genomic data produced. With the cost being down to 2,000 USD for a single human genome, sequencing dozens of individuals is a task that is feasible even for smaller project or organizations already today. However, generating the sequence is only one issue; another one is storing, managing, and analyzing it. These tasks become more and more challenging due to the sheer size of the data sets and are increasingly considered to be the most severe bottlenecks in larger genome projects. One possible countermeasure is to compress the data; compression reduces costs in terms of requiring less hard disk storage and in terms of requiring less bandwidth if data is shipped to large compute clusters for parallel analysis. Accordingly, sequence compression has recently attracted much interest in the scientific community. In this paper, we explain the different basic techniques for sequence compression, point to distinctions between different compression tasks (e.g., genome versus read compression), and present a comparison of current approaches and tools. To further stimulate progress in genome compression research, we also identify key challenges for future systems.Keywords: genome compression, read compression, survey Key messages of the article:• Overview of trends in genome compression: bit manipulation, dictionarybased, statistical, and referential compression schemes• Wide ranges of compression performance (compression ratio, compression time, memory requirements)• Comparing compression schemes for evaluation purposes is difficult• Many minor improvements in recent contributions

show abstract

“…Unfortunately, this variant is subject to a patent of IBM, AT&T, and Mitsubishi. In addition, the arithmetic coding can save 5 to 10 percent when the picture quality is not high because low quality makes the variety of AC elements smaller, and in such a case the arithmetic coding can almost reach entropy, whereas Huffman coding can reach entropy only when there exists a big variety of AC elements as explained in (Bookstein & Klein, 1993). Our paper aims to improve the compression of high quality pictures where there is no big difference between Huffman coding and arithmetic coding.…”

Section: Resultsmentioning

confidence: 99%