Multivariate Pointwise Information-Driven Data Sampling and Visualization

Abstract

With increasing computing capabilities of modern supercomputers, the size of the data generated from the scientific simulations is growing rapidly. As a result, application scientists need effective data summarization techniques that can reduce large-scale multivariate spatiotemporal data sets while preserving the important data properties so that the reduced data can answer domain-specific queries involving multiple variables with sufficient accuracy. While analyzing complex scientific events, domain experts often analyze and visualize two or more variables together to obtain a better understanding of the characteristics of the data features. Therefore, data summarization techniques are required to analyze multi-variable relationships in detail and then perform data reduction such that the important features involving multiple variables are preserved in the reduced data. To achieve this, in this work, we propose a data sub-sampling algorithm for performing statistical data summarization that leverages pointwise information theoretic measures to quantify the statistical association of data points considering multiple variables and generates a sub-sampled data that preserves the statistical association among multi-variables. Using such reduced sampled data, we show that multivariate feature query and analysis can be done effectively. The efficacy of the proposed multivariate association driven sampling algorithm is presented by applying it on several scientific data sets.

Accelerating In-Transit Co-Processing for Scientific Simulations Using Region-Based Data-Driven Analysis

“…Determining and analyzing regions of interest in three-dimensional (3D) data sets has been an important topic in many research fields, and has as such been explored in many related works [10][11][12][13][14][15][16][17]. A commonly used technique is Adaptive Mesh Refinement (AMR) [13][14][15].…”

“…A prioritization tree was constructed for the data set, which then could be used to identify interesting camera placements or to determine compression strategies for saving data to permanent storage. Some data sampling and summarization methods [16,17] use importance based on entropy metrics to prioritize the reduction of unimportant data. Here, reduced subsets of the simulation data are saved to permanent storage for post-hoc analysis; more than 99% of simulation data is removed in some applications [16].…”

“…Method [11] is processed posthoc, whereas methods [12][13][14][15] have no defined behavior of how to use the calculated importance values to compress unimportant data. Out of the methods used for comparison, only related work [10,16,17] explore using the data importance to prioritize compression or reduction of unimportant data. However, in contrast to our work, none of the methods support lossless compression or simultaneously using multiple compression and reduction methods.…”

“…Such information can be used for a wide variety of cases, including guiding the simulation, determining which time steps of the simulation to analyze or save to permanent storage, removing or reducing unimportant data, and finding interesting camera locations for co-processing purposes. Some of these cases have been explored in related work [10][11][12][13][14][15][16][17]. However, these methods have either been limited in the amount of used importance metrics [10,[12][13][14][15][16][17] or been unable to combine multiple metrics to create advanced importance metrics and to make more advanced decisions [11].…”

ECP Software Technology Capability Assessment Report

“…Adaptive sampling can be used to guide visualizations and extracts to the most important parts of the simulation, significantly reducing I/O. Figure 68 shows an adaptive sampling technique based on importance to preserve features in the data [210,211,212]. Moments-based pattern detection can be used to find rotation-invariant patterns [213,214,215].…”