Data Reduction Techniques for Simulation, Visualization and Data Analysis

Li, Samuel; Marsaglia, Nicole; Garth, Christoph; Woodring, Jonathan; Clyne, John; Childs, Hank

doi:10.1111/cgf.13336

Cited by 73 publications

(43 citation statements)

References 171 publications

(199 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While no loss of precision is applied in theory because transformations are invertible, transformations on floating-point data typically lead to rounding errors. In literature those transformations are therefore also addressed as nearlossless methods [82]. Prediction A good derivation of a data predictor allows estimating the value of adjacent data.…”

Section: Methodsmentioning

confidence: 99%

“…Relief can be achieved by using lossy compression since it allows to trade in data quality for data size. Many techniques for lossy compression have been developed with a focus on multimedia data such as audio or image data, but they can be reused for scientific data too [82]. However, losing data quality is not acceptable for all users and limits possible use cases.…”

Section: Compressionmentioning

confidence: 99%

See 1 more Smart Citation

State of the Art and Future Trends in Data Reduction for High-Performance Computing

2020

JSFI

View full text Add to dashboard Cite

Research into data reduction techniques has gained popularity in recent years as storage capacity and performance become a growing concern. This survey paper provides an overview of leveraging points found in high-performance computing (HPC) systems and suitable mechanisms to reduce data volumes. We present the underlying theories and their application throughout the HPC stack and also discuss related hardware acceleration and reduction approaches. After introducing relevant use-cases, an overview of modern lossless and lossy compression algorithms and their respective usage at the application and file system layer is given. In anticipation of their increasing relevance for adaptive and in situ approaches, dimensionality reduction techniques are summarized with a focus on non-linear feature extraction. Adaptive approaches and in situ compression algorithms and frameworks follow. The key stages and new opportunities to deduplication are covered next. An unconventional but promising method is recomputation, which is proposed at last. We conclude the survey with an outlook on future developments.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Compressionmentioning

confidence: 99%

State of the Art and Future Trends in Data Reduction for High-Performance Computing

2020

JSFI

View full text Add to dashboard Cite

show abstract

“…Moreover, many researchers (Ballester-Ripoll et al, 2018;Liang et al, 2018;Lindstrom, 2014;Tao et al, 2017c) assess the compression quality in terms of visualization-related metrics such as PSNR. More detailed discussions of the lossy compression techniques regarding the visual quality can be found in Li et al's (2018b) survey.…”

Section: Visualizationmentioning

confidence: 99%

“…In the past, lossy compressors for scientific data focused almost exclusively on data reduction for visualization. The lossy compressors used techniques directly inherited from lossy compression of images such as variations of wavelet transforms, coefficient prioritization, and vector quantization (Goldschneider, 1997; Li et al, 2018b). Lossy compressors for image processing are designed and optimized considering human perception.…”

Section: Progress In Lossy Compression Technologiesmentioning

confidence: 99%

Use cases of lossy compression for floating-point data in scientific data sets

Cappello

et al. 2019

The International Journal of High Performance Computing Applica

109

View full text Add to dashboard Cite

Architectural and technological trends of systems used for scientific computing call for a significant reduction of scientific data sets that are composed mainly of floating-point data. This article surveys and presents experimental results of currently identified use cases of generic lossy compression to address the different limitations of scientific computing systems. The article shows from a collection of experiments run on parallel systems of a leadership facility that lossy data compression not only can reduce the footprint of scientific data sets on storage but also can reduce I/O and checkpoint/restart times, accelerate computation, and even allow significantly larger problems to be run than without lossy compression. These results suggest that lossy compression will become an important technology in many aspects of high performance scientific computing. Because the constraints for each use case are different and often conflicting, this collection of results also indicates the need for more specialization of the compression pipelines.

show abstract

“…Finally, the general topic of scientific data reduction is a quite complex and active research area. This survey [34] provides an overview of this topic.…”

Section: Wavelet Transforms In Vapor Data Collectionmentioning

confidence: 99%

VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science

et al. 2019

Self Cite

View full text Add to dashboard Cite

Visualization is an essential tool for analysis of data and communication of findings in the sciences, and the Earth System Sciences (ESS) are no exception. However, within ESS, specialized visualization requirements and data models, particularly for those data arising from numerical models, often make general purpose visualization packages difficult, if not impossible, to use effectively. This paper presents VAPOR: a domain-specific visualization package that targets the specialized needs of ESS modelers, particularly those working in research settings where highly-interactive exploratory visualization is beneficial. We specifically describe VAPOR’s ability to handle ESS simulation data from a wide variety of numerical models, as well as a multi-resolution representation that enables interactive visualization on very large data while using only commodity computing resources. We also describe VAPOR’s visualization capabilities, paying particular attention to features for geo-referenced data and advanced rendering algorithms suitable for time-varying, 3D data. Finally, we illustrate VAPOR’s utility in the study of a numerically- simulated tornado. Our results demonstrate both ease-of-use and the rich capabilities of VAPOR in such a use case.

show abstract

Data Reduction Techniques for Simulation, Visualization and Data Analysis

Cited by 73 publications

References 171 publications

State of the Art and Future Trends in Data Reduction for High-Performance Computing

State of the Art and Future Trends in Data Reduction for High-Performance Computing

Use cases of lossy compression for floating-point data in scientific data sets

VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science

Contact Info

Product

Resources

About