Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets

Liang, Xin; Di, Sheng; Tao, Dingwen; Li, Sihuan; Li, Samuel; Guo, Hanqi; Chen, Zizhong; Cappello, Franck

doi:10.1109/bigdata.2018.8622520

Cited by 189 publications

(161 citation statements)

References 16 publications

Supporting

Mentioning

143

Contrasting

Order By: Relevance

“…The mathematical formula of least-squares linear regression gives the best fit solution [46]. The least-squares regression defines the values of a and b that will minimize the mean squared residual, 2 ̅̅̅ , where e is a residual:…”

Section: Pla Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Experimental Study on Machine Learning with Approximation to Data Streams

Zhang

Jiang

2019

2019 Sixth International Conference on Internet of Things: Systems, Management and Security (IOTSMS)

View full text Add to dashboard Cite

Realtidsöverföring av dataströmmar möjliggör många dataanalyser och maskininlärningsapplikationer inom områdena t.ex. massiv IoT och industriell automatisering. Stor datavolym för dessa strömmar är en betydande börda eller omkostnad inte bara för transportnätet utan också för motsvarande applikationsservrar. Därför fokuserar forskare och forskare om att minska mängden data som behövs för att överföras via datakomprimeringar och approximationer. Datakomprimeringstekniker som förlustkomprimering kan minska datavolymen betydligt med priset för datainformation. Samtidigt är datakomprimering mycket beroende av motsvarande applikationer. Men när du använder dekomprimerade data i en viss dataanalysapplikation som maskininlärning, kan resultaten påverkas på grund av informationsförlusten. I denna artikel gjorde författaren en studie om effekterna av datakomprimering på maskininlärningsapplikationerna.

show abstract

Section: Pla Methodsmentioning

confidence: 99%

“…High-performance compression techniques are essential to today's scientific research due to the massive volume of data with limited storage space and limited I/O bandwidth energy to access [2]. Mainstream compression modes are basically lossless compression and lossy compression.…”

Section: Introductionmentioning

confidence: 99%

Experimental Study on Machine Learning with Approximation to Data Streams

Zhang

Jiang

2019

2019 Sixth International Conference on Internet of Things: Systems, Management and Security (IOTSMS)

View full text Add to dashboard Cite

show abstract

“…This model tries to predict each data point as accurately as possible based on its neighborhood in spatial or temporal dimension and then shrinks the data size by some coding algorithm such as data quantization [62] and bit-plane truncation. A typical example compressor is SZ [44], which involves four compression steps: (1) data prediction, (2) linear-scaling quantization, (3) entropy-encoding, and (4) lossless compression. The errors are introduced and controlled at step (2).…”

Section: Data Compression Techniquesmentioning

confidence: 99%

Full-state quantum circuit simulation by using data compression

Dasgupta

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

Quantum circuit simulations are critical for evaluating quantum algorithms and machines. However, the number of state amplitudes required for full simulation increases exponentially with the number of qubits. In this study, we leverage data compression to reduce memory requirements, trading computation time and fidelity for memory space. Specifically, we develop a hybrid solution by combining the lossless compression and our tailored lossy compression method with adaptive error bounds at each timestep of the simulation. Our approach optimizes for compression speed and makes sure that errors due to lossy compression are uncorrelated, an important property for comparing simulation output with physical machines. Experiments show that our approach reduces the memory requirement of simulating the 61-qubit Grover's search algorithm from 32 exabytes to 768 terabytes of memory on Argonne's Theta supercomputer using 4,096 nodes. The results suggest that our techniques can increase the simulation size by 2∼16 qubits for general quantum circuits.

show abstract

“…Unlike traditional compression methods used on DNNs, we perform error-bounded lossy compression on the pruned weights, an approach that can significantly reduce the data size while restricting the loss of inference accuracy. Specifically, we adapt the SZ lossy compression framework developed by us previously [11,28,39] to fit the context of DNN compression. In this compression framework, each data point's value would be predicted based on its neighboring data points by an adaptive, best-fit prediction method (either a Lorenzo predictor or linear regression-based predictor [28]).…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, we adapt the SZ lossy compression framework developed by us previously [11,28,39] to fit the context of DNN compression. In this compression framework, each data point's value would be predicted based on its neighboring data points by an adaptive, best-fit prediction method (either a Lorenzo predictor or linear regression-based predictor [28]). Then, each floating-point weight value would be converted to an integer number by a linearscaling quantization based on the difference between the real value and predicted value and a specific error bound.…”

Section: Introductionmentioning

confidence: 99%

DeepSZ

Jin

Liang

et al. 2019

Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing

Self Cite

View full text Add to dashboard Cite

Today's deep neural networks (DNNs) are becoming deeper and wider because of increasing demand on the analysis quality and more and more complex applications to resolve. The wide and deep DNNs, however, require large amounts of resources (such as memory, storage, and I/O), significantly restricting their utilization on resource-constrained platforms. Although some DNN simplification methods (such as weight quantization) have been proposed to address this issue, they suffer from either low compression ratios or high compression errors, which may introduce an expensive fine-tuning overhead (i.e., a costly retraining process for the target inference accuracy). In this paper, we propose DeepSZ: an accuracyloss expected neural network compression framework, which involves four key steps: network pruning, error bound assessment, optimization for error bound configuration, and compressed model generation, featuring a high compression ratio and low encoding time. The contribution is threefold. (1) We develop an adaptive approach to select the feasible error bounds for each layer. (2) We build a model to estimate the overall loss of inference accuracy based on the inference accuracy degradation caused by individual decompressed layers.(3) We develop an efficient optimization algorithm to determine the best-fit configuration of error bounds in order to maximize the compression ratio under the user-set inference accuracy constraint. Experiments show that DeepSZ can compress AlexNet and VGG-16 on the ImageNet dataset by a compression ratio of 46× and 116×, respectively, and compress LeNet-300-100 and LeNet-5 on the MNIST dataset by a compression ratio of 57× and 56×, respectively, with only up to 0.3% loss of inference accuracy. Compared with other state-of-the-art methods, DeepSZ can improve the compression ratio by up to 1.43×, the DNN encoding performance by up to 4.0× with four V100 GPUs, and the decoding performance by up to 6.2×.

show abstract

Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets

Cited by 189 publications

References 16 publications

Experimental Study on Machine Learning with Approximation to Data Streams

Experimental Study on Machine Learning with Approximation to Data Streams

Full-state quantum circuit simulation by using data compression

DeepSZ

Contact Info

Product

Resources

About