Optimal Compressed Sensing and Reconstruction of Unstructured Mesh Datasets

Salloum, Maher; Fabian, Nathan D.; Hensinger, David M.; Lee, Jina; Allendorf, Elizabeth M.; Bhagatwala, Ankit; Blaylock, Myra L.; Chen, Jacqueline H.; Templeton, Jeremy Alan; Tezaur, Irina Kalashnikova

doi:10.1007/s41019-017-0042-4

Cited by 11 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, the method requires extra handling of high‐error points with a regression algorithm. Salloum et al [SFH*18] used a compressed sensing approach to compress 2D unstructured grid data, which requires an iterative and optimization process to decompress the data. Salloum et al [SKJ*20] further explored using Alpert multi‐wavelets for compressing unstructured grid data.…”

Section: Related Workmentioning

confidence: 99%

A Prediction‐Traversal Approach for Compressing Scientific Data on Unstructured Meshes with Bounded Error

Ren,

Liang,

Guo

2024

Computer Graphics Forum

View full text Add to dashboard Cite

We explore an error‐bounded lossy compression approach for reducing scientific data associated with 2D/3D unstructured meshes. While existing lossy compressors offer a high compression ratio with bounded error for regular grid data, methodologies tailored for unstructured mesh data are lacking; for example, one can compress nodal data as 1D arrays, neglecting the spatial coherency of the mesh nodes. Inspired by the SZ compressor, which predicts and quantizes values in a multidimensional array, we dynamically reorganize nodal data into sequences. Each sequence starts with a seed cell; based on a predefined traversal order, the next cell is added to the sequence if the current cell can predict and quantize the nodal data in the next cell with the given error bound. As a result, one can efficiently compress the quantized nodal data in each sequence until all mesh nodes are traversed. This paper also introduces a suite of novel error metrics, namely continuous mean squared error (CMSE) and continuous peak signal‐to‐noise ratio (CPSNR), to assess compression results for unstructured mesh data. The continuous error metrics are defined by integrating the error function on all cells, providing objective statistics across nonuniformly distributed nodes/cells in the mesh. We evaluate our methods with several scientific simulations ranging from ocean‐climate models and computational fluid dynamics simulations with both traditional and continuous error metrics. We demonstrated superior compression ratios and quality than existing lossy compressors.

show abstract

Section: Related Workmentioning

confidence: 99%

A Prediction‐Traversal Approach for Compressing Scientific Data on Unstructured Meshes with Bounded Error

Ren,

Liang,

Guo

2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…In situ analysis algorithms may transform data into reduced representations or surrogate models in order to mitigate large data size, high dimensionality, or long computation times. Low-rank approximation (Austin et al, 2016), statistical summarization (Biswas et al, 2018;Dutta et al, 2017;Hazarika et al, 2018;Lawrence et al, 2017;Lohrmann et al, 2017;Thompson et al, 2011), topological segmentation (Gyulassy et al, 2012(Gyulassy et al, , 2019Landge et al, 2014;Weber, 2013, 2014), wavelet transformation (Li et al, 2017;Salloum et al, 2018), lossy compression (Brislawn et al, 2012;Di and Cappello, 2016;Lindstrom, 2014), geometric modeling (Nashed et al, 2019; Peterka et al, 2018), and feature detection (Guo et al, 2017) may be used to generate reduced or surrogate models.…”

Section: Analysis Algorithmsmentioning

confidence: 99%

“…Research is required to modify existing post hoc algorithms and develop new in situ algorithms to satisfy the needs of modern use cases on emerging system architectures that can feature massive scale, many cores, deep memory hierarchies, or embedded lightweight edge devices. Examples of such algorithms include reduced representations and low-rank approximations (Austin et al, 2016), statistical (Biswas et al, 2018;Dutta et al, 2017;Hazarika et al, 2018;Thompson et al, 2011), topological (Gyulassy et al, 2012(Gyulassy et al, , 2019Landge et al, 2014;Weber, 2013, 2014), wavelets (Li et al, 2017;Salloum et al, 2018), compression (Brislawn et al, 2012;Di and Cappello, 2016;Lindstrom, 2014), and feature detection (Guo et al, 2017) methods. Surrogate models and multifidelity models can be geometric (Nashed et al, 2019;Peterka et al, 2018), statistical (Lawrence et al, 2017;Lohrmann et al, 2017), or neural network (He et al, 2019).…”

Section: In Situ Algorithmsmentioning

confidence: 99%

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

Peterka

Bard

Bennett

et al. 2020

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

show abstract

“…Compressed sensing techniques, the subject of this paper, can also be implemented in a lossy manner by including quantization in the process [29,26], or by combining it with wavelets [27].…”

Section: A Comparison Of Compressed Sensing and Sparse Recovery Algormentioning

confidence: 99%

A Comparison of Compressed Sensing and Sparse Recovery Algorithms Applied to Simulation Data

Fan

Kamath

2016

Stat., optim. inf. comput.

View full text Add to dashboard Cite

The move toward exascale computing for scientific simulations is placing new demands on compression techniques. It is expected that the I/O system will not be able to support the volume of data that is expected to be written out. To enable quantitative analysis and scientific discovery, we are interested in techniques that compress high-dimensional simulation data and can provide perfect or near-perfect reconstruction. In this paper, we explore the use of compressed sensing (CS) techniques to reduce the size of the data before they are written out. Using large-scale simulation data, we investigate how the sufficient sparsity condition and the contrast in the data affect the quality of reconstruction and the degree of compression. We provide suggestions for the practical implementation of CS techniques and compare them with other sparse recovery methods. Our results show that despite longer times for reconstruction, compressed sensing techniques can provide near perfect reconstruction over a range of data with varying sparsity. Background and MotivationThe analysis of data from scientific simulations is typically done by writing out the variables of interest at each time step and analyzing them at a later time. For simulations run on massively parallel machines, the volume of data written out frequently approaches terabytes and beyond. In some problems, writing out the data could be more time consuming than the computations done at each time step. As we move to exascale computing, it is expected that the I/O subsystem will not be able to provide the needs of the simulations [8]. Instead of writing out the data for analysis, it has been proposed that the analysis algorithms be moved in situ, and only the analysis results, which are hopefully much smaller, be written out. This is a viable option that works when we know the analysis algorithm we want to use and its parameters. However, in many cases, the choice of analysis algorithms and their parameters may depend on what we find as we analyze the simulation output. This is especially true when the motivation for the analysis is scientific understanding and discovery. In such cases, the analysis cannot be done in situ and alternate approaches have to be considered to address this problem of limited I/O bandwidth.One such approach is to reduce the size of the simulation output by compressing the data before they are written out. There are several ways of compressing data, ranging from the traditional compression schemes, such as JPEG, that are used for images, to others that are based on clustering techniques from data mining. For the data set and problem considered in this paper, we require loss-less compression techniques that can be applied to data that are irregularly spaced in two or three dimensions and are distributed across multiple processors (or files). We propose

show abstract

Optimal Compressed Sensing and Reconstruction of Unstructured Mesh Datasets

Cited by 11 publications

References 44 publications

A Prediction‐Traversal Approach for Compressing Scientific Data on Unstructured Meshes with Bounded Error

A Prediction‐Traversal Approach for Compressing Scientific Data on Unstructured Meshes with Bounded Error

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

A Comparison of Compressed Sensing and Sparse Recovery Algorithms Applied to Simulation Data

Contact Info

Product

Resources

About