FPC: A High-Speed Compressor for Double-Precision Floating-Point Data

Burtscher, Martin; Ratanaworabhan, Paruj

doi:10.1109/tc.2008.131

Cited by 217 publications

(150 citation statements)

References 17 publications

Supporting

Mentioning

137

Contrasting

Unclassified

Order By: Relevance

“…FPC is one of prediction-based lossless compression techniques for general-purpose FP applications [9]. FPC employs a hybrid predictor of FCM (finite context method) [16] and DFCM (differential FCM) [17].…”

Section: Preliminariesmentioning

confidence: 99%

“…Meanwhile we measured hit rates in bits of the hybrid predictor of FPC for concrete FP data sets of numerical simulations presented in [9] as a preliminary step. As shown in Fig.…”

Section: Preliminariesmentioning

confidence: 99%

“…For example, FPC encodes the number of succeeding zero bytes counting from the sign bit [9]; in Isenburg's technique, a FP number is compressed efficiently when the true and the predicted values have the same sign and the difference between their exponent is within 3 [5]. Thus we in this paper evaluate predictors by counting number of leading zero bytes (NLZ) in the difference.…”

Section: Parallel Predictorsmentioning

confidence: 99%

“…The parenthetic values written with FCM and DFCM are size parameters for hash tables mentioned in the previous section, that is for example, "FCM (9)" has a hash table with M = 2 9 entries. Larger hash table usually presents better compression ratio in both FCM and DFCM [9]. "brain", "comet", "control",and "plasma" are the names of data sets resulted from scientific applications presented in [9]; "EM" is a set of FP numbers resulted from a Finite-Difference Time-Domain (FDTD) based simulator for electromagnetic fields [21], [22] implemented by one of the autors [23].…”

Section: Parallel Predictorsmentioning

confidence: 99%

“…FPC is one of general-purpose lossless compression techniques for double-precision FP numbers [9] (hereafter we simply refer to a double-precision FP number as double). FPC can well compress FP data of wide applications, such as message passing, sattelite observation, and numerical simulations.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Lossless Compression of Double-Precision Floating-Point Data for Numerical Simulations: Highly Parallelizable Algorithms for GPU Computing

Ohara

Yamaguchi

2012

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYIn numerical simulations using massively parallel computers like GPGPU (General-Purpose computing on Graphics Processing Units), we often need to transfer computational results from external devices such as GPUs to the main memory or secondary storage of the host machine. Since size of the computation results is sometimes unacceptably large to hold them, it is desired that the data is compressed and stored. In addition, considering overheads for transferring data between the devices and host memories, it is preferable that the data is compressed in a part of parallel computation performed on the devices. Traditional compression methods for floating-point numbers do not always show good parallelism. In this paper, we propose a new compression method for massivelyparallel simulations running on GPUs, in which we combine a few successive floating-point numbers and interleave them to improve compression efficiency. We also present numerical examples of compression ratio and throughput obtained from experimental implementations of the proposed method runnig on CPUs and GPUs.

show abstract

Section: Preliminariesmentioning

confidence: 99%

“…Meanwhile we measured hit rates in bits of the hybrid predictor of FPC for concrete FP data sets of numerical simulations presented in [9] as a preliminary step. As shown in Fig.…”

Section: Preliminariesmentioning

confidence: 99%

Section: Parallel Predictorsmentioning

confidence: 99%

Section: Parallel Predictorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Lossless Compression of Double-Precision Floating-Point Data for Numerical Simulations: Highly Parallelizable Algorithms for GPU Computing

Ohara

Yamaguchi

2012

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

In Situ Data Compression Algorithm for Detailed Numerical Simulation of Liquid Metal Filtration through Regularly Structured Porous Media

et al. 2013

View full text Add to dashboard Cite

Filtration of metal melts is a common practice used in order to produce materials that meet high requirements with respect to strength and toughness. Numerical simulations of the filtration process not only help to investigate the involved mechanisms but can also contribute to process optimization. Owing to the presence of a wide range of spatial and temporal scales which have to be resolved, these simulations are generally carried out on extremely refined computational grids with high temporal resolution using massively parallelized computation. However, the amount of data resulting from such computations can easily exceed dozens of terabytes, which imposes a severe hindrance on the storage, visualization, and analysis of the solutions. A promising approach for addressing this problem is in situ data compression, where data is already compressed during the simulation run on the supercomputer rather than as a post-processing step. In this article, a detailed numerical simulation is described for the liquid metal flow through a foam-like filter structure using the lattice Boltzmann method. A novel in situ data compression procedure is investigated that, while lossy, can achieve arbitrary precision to meet an error bound specified by the user. The efficiency and accuracy of the algorithm are evaluated by statistical analysis and by comparing the trajectories of tracer particles in uncompressed and compressed flow fields. The results demonstrate that, depending on the desired accuracy, compression rates of as low as 20% of the original data size can be achieved.

show abstract

ISABELA for effective in situ compression of scientific data

Lakshminarasimhan

Shah

Ethier

et al. 2012

Concurrency and Computation

View full text Add to dashboard Cite

Exploding dataset sizes from extreme-scale scientific simulations necessitates efficient data management and reduction schemes to mitigate I/O costs. With the discrepancy between I/O bandwidth and computational power, scientists are forced to capture data infrequently, thereby making data collection an inherently lossy process. Although data compression can be an effective solution, the random nature of real-valued scientific datasets renders lossless compression routines ineffective. These techniques also impose significant overhead during decompression, making them unsuitable for data analysis and visualization, which require repeated data access.To address this problem, we propose an effective method for In situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data that is widely regarded as effectively incompressible. With ISABELA, we apply a pre-conditioner to seemingly random and noisy data along spatial resolution to achieve an accurate fitting model that guarantees a > 0.99 correlation with the original data. We further take advantage of temporal patterns in scientific data to compress data by 85%, while introducing only a negligible overhead on simulations in terms of runtime. ISABELA significantly outperforms existing lossy compression methods, such as wavelet compression, in terms of data reduction and accuracy.We extend upon our previous paper by additionally building a communication-free, scalable parallel storage framework on top of ISABELA-compressed data that is ideally suited for extreme-scale analytical processing. The basis for our storage framework is an inherently local decompression method (it need not decode the entire data), which allows for random access decompression and low-overhead task division that can be exploited over heterogeneous architectures. Furthermore, analytical operations such as correlation and query processing run quickly and accurately over data in the compressed space. 525In situ data processing-or processing the data in-tandem with the simulation by utilizing either the same compute nodes or the staging nodes-is emerging as a promising approach to address the I/O bottleneck [1]. To complement existing approaches, we introduce an effective method for In situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data [2]. ISABELA is particularly designed for compressing spatio-temporal scientific data that is characterized as being inherently noisy and random-like and thus commonly believed to be incompressible [3]. In fact, the majority of the lossless compression techniques [4,5] are not only computationally intensive and therefore hardly suitable for in situ processing but also are unable to reduce such data by more than 10% of its original size (see Section 3).The intuition behind ISABELA stems from the following three observations. First, while being almost random and noisy in its natural form-when sorted-scientific data exhibits a very strong signal-to-noise ratio due to its monotonic and smooth behavior in its sorted form....

show abstract

FPC: A High-Speed Compressor for Double-Precision Floating-Point Data

Cited by 217 publications

References 17 publications

Lossless Compression of Double-Precision Floating-Point Data for Numerical Simulations: Highly Parallelizable Algorithms for GPU Computing

Lossless Compression of Double-Precision Floating-Point Data for Numerical Simulations: Highly Parallelizable Algorithms for GPU Computing

In Situ Data Compression Algorithm for Detailed Numerical Simulation of Liquid Metal Filtration through Regularly Structured Porous Media

ISABELA for effective in situ compression of scientific data

Contact Info

Product

Resources

About