FRaZ: A Generic High-Fidelity Fixed-Ratio Lossy Compression Framework for Scientific Floating-point Data

Underwood, Robert; Di, Sheng; Calhoun, Jon C.; Cappello, Franck

doi:10.1109/ipdps47924.2020.00065

Cited by 36 publications

(17 citation statements)

References 27 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although ZFP [8] provides a fix-rate mode (i.e., fix-compression-ratio), its compression quality is significantly worse than that of the error-bounding mode, as verified by Underwood et al [33]. This requires ZFP users to conduct a trial-and-error approach to obtain a target compression ratio given a scientific dataset [12], [33]. The lack of compression ratio estimation and data quality estimation make lossy compression users rely on the trial-anderror technique for compression parameter optimization [12], [13].…”

Section: B Error-bounded Lossy Compressionmentioning

confidence: 87%

“…SZ -a popular prediction-based lossy compressor, for example, cannot perform compression based on a given target compression ratio. Although ZFP [8] provides a fix-rate mode (i.e., fix-compression-ratio), its compression quality is significantly worse than that of the error-bounding mode, as verified by Underwood et al [33]. This requires ZFP users to conduct a trial-and-error approach to obtain a target compression ratio given a scientific dataset [12], [33].…”

Section: B Error-bounded Lossy Compressionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Prediction-Based Lossy Compression Dramatically via Ratio-Quality Modeling

Jin¹,

Di²,

Tian³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Error-bounded lossy compression is one of the most effective techniques for reducing scientific data sizes. However, the traditional trial-and-error approach used to configure lossy compressors for finding the optimal trade-off between reconstructed data quality and compression ratio is prohibitively expensive. To resolve this issue, we develop a general-purpose analytical ratio-quality model based on the prediction-based lossy compression framework, which can effectively foresee the reduced data quality and compression ratio, as well as the impact of lossy compressed data on post-hoc analysis quality. Our analytical model significantly improves the prediction-based lossy compression in three use-cases: (1) optimization of predictor by selecting the best-fit predictor; (2) memory compression with a target ratio; and (3) in-situ compression optimization by fine-grained tuning error-bounds for various data partitions. We evaluate our analytical model on 10 scientific datasets, demonstrating its high accuracy (93.47% accuracy on average) and low computational cost (up to 18.7× lower than the trialand-error approach) for estimating the compression ratio and the impact of lossy compression on post-hoc analysis quality. We also verify the high efficiency of our ratio-quality model using different applications across the three use-cases. In addition, our experiment demonstrates that our modeling based approach reduces the time to store the 3D RTM data with HDF5 by up to 3.4× with 128 CPU cores over the traditional solution.

show abstract

Section: B Error-bounded Lossy Compressionmentioning

confidence: 87%

Section: B Error-bounded Lossy Compressionmentioning

confidence: 99%

Improving Prediction-Based Lossy Compression Dramatically via Ratio-Quality Modeling

Jin¹,

Di²,

Tian³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, if we repeat our experiment for a larger number of iterations, we achieve a value within 5-10% of peak performance. The high-cost of autotuning for larger percentages and larger number of iterations could be amortized over several compressor configurations as recent work has shows that for time-series data -e.g., fields in a scientific application -a optimal compressor configuration remains mostly constant through time [32]. We find, for example, that across all 48 time-steps of a field of the Hurricane Isabel dataset, an average of 80% of the autotuning runs result in two top configurations.…”

Section: F Autotuning Block Size and Vector Lengthmentioning

confidence: 89%

SIMD Lossy Compression for Scientific Data

Griffin¹,

Tian²,

Di³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Modern HPC applications produce increasingly large amounts of data, which limits the performance of current extreme-scale systems. Data reduction techniques, such as lossy compression, help to mitigate this issue by decreasing the size of data generated by these applications. SZ, a current state-of-theart lossy compressor, is able to achieve high compression ratios, but the prediction/quantization methods used introduce dependencies which prevent parallelizing this step of the compression. Recent work proposes a parallel dual prediction/quantization algorithm for GPUs which removes these dependencies. However, some HPC systems and applications do not use GPUs, and could still benefit from the fine-grained parallelism of this method. Using the dual-quantization technique, we implement and optimize a SIMD vectorized CPU version of SZ, and create a heuristic for selecting the optimal block size and vector length. We also investigate the effect of non-zero block padding values to decrease the number of unpredictable values along compression block borders. We measure performance of vecSZ against an O3 optimized CPU version of SZ using dual-quantization, pSZ, as well as SZ-1.4. We evaluate our vectorized version, vecSZ, on the Intel Skylake and AMD Rome architectures using real-world scientific datasets. We find that applying alternative padding reduces the number of outliers by 100% for some configurations. Our implementation also results in up to 32% improvement in rate-distortion and up to 15× speedup over SZ-1.4, achieving a prediction and quantization bandwidth in excess of 3.4 GB/s.

show abstract

“…] @2.1.11.1 lossy compressor ZFP [4] @0.5.5 lossy compressor MGARD [6] @0.1.0 lossy compressor gstat [22] @2.0-7 obtain variogram range numpy [23] @1.21.1 polyfit function to graph the curves Libpressio [24] @0.70.0 compress and measure the data TABLE I COMPRESSORS AND SOFTWARE USED FOR THE STUDY latest available on Spack from a selection of leading errorbounded lossy compressors. We use the absolute error bound because it is supported by each of the leading error-bounded lossy compressors.…”

Section: Each Lossy Compressor In Tablementioning

confidence: 99%

Exploring Lossy Compressibility through Statistical Correlations of Scientific Datasets

Krasowska¹,

Bessac²,

Underwood³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Lossy compression plays a growing role in scientific simulations where the cost of storing their output data can span terabytes. Using error bounded lossy compression reduces the amount of storage for each simulation; however, there is no known bound for the upper limit on lossy compressibility. Correlation structures in the data, choice of compressor and error bound are factors allowing larger compression ratios and improved quality metrics. Analyzing these three factors provides one direction towards quantifying lossy compressibility. As a first step, we explore statistical methods to characterize the correlation structures present in the data and their relationships, through functional models, to compression ratios. We observed a relationship between compression ratios and statistics summarizing correlation structure of the data, which are a first step towards evaluating the theoretical limits of lossy compressibility used to eventually predict compression performance and adapt compressors to correlation structures present in the data.

show abstract

FRaZ: A Generic High-Fidelity Fixed-Ratio Lossy Compression Framework for Scientific Floating-point Data

Cited by 36 publications

References 27 publications

Improving Prediction-Based Lossy Compression Dramatically via Ratio-Quality Modeling

Improving Prediction-Based Lossy Compression Dramatically via Ratio-Quality Modeling

SIMD Lossy Compression for Scientific Data

Exploring Lossy Compressibility through Statistical Correlations of Scientific Datasets

Contact Info

Product

Resources

About