Information Guided Data Sampling and Recovery Using Bitmap Indexing

Wei, Tzu-Hsuan; Dutta, Soumya; Shen, Han‐Wei

doi:10.1109/pacificvis.2018.00016

Cited by 16 publications

(10 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chaudhur et al [20,21] and Lee et al [12] employed distribution to conduct efficient data query and visualization. Wei et al [13,14] used bitmap index to efficiently support both single and multivariant distribution queries for data analysis. Chen et al [22] applied Gaussian distribution to model the uncertainty of the pathline in time-varying flow field datasets.…”

Section: Related Work 21 Distribution-based Large Data Processing and Analysismentioning

confidence: 99%

“…Our data-parallel primitive-based algorithms can facilitate the data process that is run on supercomputers with different types of computing nodes. In this work, we propose parallel multi-set distribution modeling algorithms for multi-variant histogram [3,[12][13][14][15] and GMM [1,2,4,[15][16][17] modeling, because these are the most popular non-parametric and parametric distribution representations in the scientific data modeling, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient and Portable Distribution Modeling for Large-Scale Scientific Data Processing with Data-Parallel Primitives

2021

View full text Add to dashboard Cite

The use of distribution-based data representation to handle large-scale scientific datasets is a promising approach. Distribution-based approaches often transform a scientific dataset into many distributions, each of which is calculated from a small number of samples. Most of the proposed parallel algorithms focus on modeling single distributions from many input samples efficiently, but these may not fit the large-scale scientific data processing scenario because they cannot utilize computing resources effectively. Histograms and the Gaussian Mixture Model (GMM) are the most popular distribution representations used to model scientific datasets. Therefore, we propose the use of multi-set histogram and GMM modeling algorithms for the scenario of large-scale scientific data processing. Our algorithms are developed by data-parallel primitives to achieve portability across different hardware architectures. We evaluate the performance of the proposed algorithms in detail and demonstrate use cases for scientific data processing.

show abstract

Section: Related Work 21 Distribution-based Large Data Processing and Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Efficient and Portable Distribution Modeling for Large-Scale Scientific Data Processing with Data-Parallel Primitives

2021

View full text Add to dashboard Cite

show abstract

“…Then, the entropy is obtained as

{double-struckH}_{scriptP} false(A false) = - \sum_{b = 0}^{N_{bins} - 1} h false(b false) l o g_{2} h false(b false) .

We normalize the values to be independent from the histogram size according to Wei et al . [WDS18] with

{\bar{H}}_{scriptP} false(A false) = \frac{2^{H_{P} (A)}}{N_{bins}} .

…”

Section: Stochastic Samplingmentioning

confidence: 99%

Stochastic Volume Rendering of Multi‐Phase SPH Data

Max

Rapp

Dachsbacher

2020

Computer Graphics Forum

View full text Add to dashboard Cite

In this paper, we present a novel method for the direct volume rendering of large smoothed‐particle hydrodynamics (SPH) simulation data without transforming the unstructured data to an intermediate representation. By directly visualizing the unstructured particle data, we avoid long preprocessing times and large storage requirements. This enables the visualization of large, time‐dependent, and multivariate data both as a post‐process and in situ. To address the computational complexity, we introduce stochastic volume rendering that considers only a subset of particles at each step during ray marching. The sample probabilities for selecting this subset at each step are thereby determined both in a view‐dependent manner and based on the spatial complexity of the data. Our stochastic volume rendering enables us to scale continuously from a fast, interactive preview to a more accurate volume rendering at higher cost. Lastly, we discuss the visualization of free‐surface and multi‐phase flows by including a multi‐material model with volumetric and surface shading into the stochastic volume rendering.

show abstract

“…A stratified random sampling based scheme was proposed by Woodring et al [ 24 ] where the authors used cosmology simulation as their application and enabled interactive visualization of the large-scale data. Using bitmap indices and information entropy, Wei et al [ 23 ] extended the standard stratified random sampling for in situ data reduction. Previous to this work, Su et al [ 25 ] used bit map indices for sampling to enable fast user queries on the datasets.…”

Section: Related Workmentioning

confidence: 99%

“…In this work, we propose a multivariate data summarization technique that selects a small fraction of the original data set (i.e., subsamples the data) considering multiple variables simultaneously and picks data points with higher fidelity from the regions where the selected variables show a strong association. The benefits of sampling-based data representations for a single variable has been shown by the researchers in the past [ 22 , 23 , 24 , 25 ]. In this work, we pursue a novel sampling-based data summarization scheme and extend the sampling capability to the multi-variable domain.…”

Section: Introductionmentioning

confidence: 99%

Multivariate Pointwise Information-Driven Data Sampling and Visualization

Dutta

Biswas

Ahrens

2019

Entropy

Self Cite

View full text Add to dashboard Cite

With increasing computing capabilities of modern supercomputers, the size of the data generated from the scientific simulations is growing rapidly. As a result, application scientists need effective data summarization techniques that can reduce large-scale multivariate spatiotemporal data sets while preserving the important data properties so that the reduced data can answer domain-specific queries involving multiple variables with sufficient accuracy. While analyzing complex scientific events, domain experts often analyze and visualize two or more variables together to obtain a better understanding of the characteristics of the data features. Therefore, data summarization techniques are required to analyze multi-variable relationships in detail and then perform data reduction such that the important features involving multiple variables are preserved in the reduced data. To achieve this, in this work, we propose a data sub-sampling algorithm for performing statistical data summarization that leverages pointwise information theoretic measures to quantify the statistical association of data points considering multiple variables and generates a sub-sampled data that preserves the statistical association among multi-variables. Using such reduced sampled data, we show that multivariate feature query and analysis can be done effectively. The efficacy of the proposed multivariate association driven sampling algorithm is presented by applying it on several scientific data sets.

show abstract

Information Guided Data Sampling and Recovery Using Bitmap Indexing

Cited by 16 publications

References 28 publications

Efficient and Portable Distribution Modeling for Large-Scale Scientific Data Processing with Data-Parallel Primitives

Efficient and Portable Distribution Modeling for Large-Scale Scientific Data Processing with Data-Parallel Primitives

Stochastic Volume Rendering of Multi‐Phase SPH Data

Multivariate Pointwise Information-Driven Data Sampling and Visualization

Contact Info

Product

Resources

About