Lossless asymmetric single instruction multiple data codec

Abstract

This report introduces a new lossless asymmetric single instruction multiple data codec designed for extremely efficient decompression of large satellite images. A throughput in excess of 3GB/s allows decompression to proceed in parallel with asynchronous transfers from fast block devices such as disk arrays. This is made possible by a simple and fast single instruction multiple data entropy coder that removes leading null bits. Our main contribution is a new approach for vectorized prediction and encoding. Unlike previous approaches that treat the entropy coder as a black box, we account for its properties in the design of the predictor. The resulting compressed stream is 1.2 to 1.5 times as large as JPEG-2000, but can be decompressed 100 times as quickly - even faster than copying uncompressed data in memory. Applications include streaming decompression for out of core visualization. To the best of our knowledge, this is the first entirely vectorized algorithm for lossless compression

Decoding billions of integers per second through vectorization

“…Then, a single call to unpack5 8 or unpack4 8 decodes m × 8 rather than just eight integers. Recent x86 processors have SIMD instructions that operate on vectors of four 32-bit integers (m = 4) [48,49,50]. We can use these instructions to achieve a better decoding speed.…”