Optimal Compression of Floating-Point Astronomical Images Without Significant Loss of Information

Abstract

ABSTRACT. We describe a compression method for floating-point astronomical images that gives compression ratios of 6-10 while still preserving the scientifically important information in the image. The pixel values are first preprocessed by quantizing them into scaled integer intensity levels, which removes some of the uncompressible noise in the image. The integers are then losslessly compressed using the fast and efficient Rice algorithm and stored in a portable FITS format file. Quantizing an image more coarsely gives greater image compression, but it also increases the noise and degrades the precision of the photometric and astrometric measurements in the quantized image. Dithering the pixel values during the quantization process can greatly improve the precision of measurements in the images. This is especially important if the analysis algorithm relies on the mode or the median, which would be similarly quantized if the pixel values are not dithered. We perform a series of experiments on both synthetic and real astronomical CCD images to quantitatively demonstrate that the magnitudes and positions of stars in the quantized images can be measured with the predicted amount of precision. In order to encourage wider use of these image compression methods, we have made available a pair of general-purpose image compression programs, called fpack and funpack, which can be used to compress any FITS format image.

Compression of interferometric radio-astronomical data

“…FITS compression uses exponential quantization (the mantissa is rounded). It supports subtractive dithering (Pence et al 2010). AIPS and BITSHUFFLE do not use dithering.…”

“…The RF and AF-normalization schemes introduced in this work also solve this issue. The lack of dithering in AIPS compression might lead to a systematic bias (Pence et al 2010). This is only an issue in the case of high SNR.…”

Real-time processing of the imaging data from the network of Las Cumbres Observatory Telescopes using BANZAI

“…"Processed" files are produced at the end of each night, using master calibration files from the same day as the observations (if available). All of our files are provided in Rice-compressed, "fpacked" 8,9 format.…”

An assessment of the use of ocean gliders to undertake acoustic measurements of zooplankton: the distribution and density of Antarctic krill (Euphausia superba) in the Weddell Sea.

“…Presently, the transmission of all the acoustic data by satellite is not feasible due to transmission rates that would require the glider to be at the surface for long periods, therefore at the mercy of strong surface currents, as well as the cost of such high-volume transfer. Key areas for future development in glider and other autonomous systems are improved on-board processing, compression, and transmission of data (Pence et al 2010). The principal challenge of such work is balancing the computing performance needed to algorithmically analyze volumes of acoustic data, and the power constraints of long-duration platforms.…”