Homogeneity guided probabilistic data summaries for analysis and visualization of large-scale data sets

Dutta, Soumya; Woodring, Jonathan; Shen, Han‐Wei; Chen, Jen‐Ping; Ahrens, James

doi:10.1109/pacificvis.2017.8031585

Cited by 25 publications

(17 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This technique is called regular sampling. Regular sampling does not consider any data properties while selecting samples and due to the regular nature of sample selection, it produces artifacts and discontinuities during sample-based visual analysis [ 17 , 22 ].…”

Section: Methodsmentioning

confidence: 99%

“…As a result, a primary requirement of such summarization process will be to preserve the salient regions (i.e., regions where multiple variables demonstrate specific trends) of the data with high fidelity such that query-driven multivariate analysis and visualization can be done efficiently using the reduced subset of the data. Scientists in the past have proposed several data reduction approaches such as distribution-based [ 6 , 10 , 15 , 16 , 17 ], image-based [ 3 , 5 ], wavelet compression based [ 18 , 19 , 20 ], data compression by functional approximations [ 21 ] etc. However, a majority of these techniques perform data reduction on each variable individually and overlook the relationship among variables.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multivariate Pointwise Information-Driven Data Sampling and Visualization

Dutta

Biswas

Ahrens

2019

Entropy

Self Cite

View full text Add to dashboard Cite

With increasing computing capabilities of modern supercomputers, the size of the data generated from the scientific simulations is growing rapidly. As a result, application scientists need effective data summarization techniques that can reduce large-scale multivariate spatiotemporal data sets while preserving the important data properties so that the reduced data can answer domain-specific queries involving multiple variables with sufficient accuracy. While analyzing complex scientific events, domain experts often analyze and visualize two or more variables together to obtain a better understanding of the characteristics of the data features. Therefore, data summarization techniques are required to analyze multi-variable relationships in detail and then perform data reduction such that the important features involving multiple variables are preserved in the reduced data. To achieve this, in this work, we propose a data sub-sampling algorithm for performing statistical data summarization that leverages pointwise information theoretic measures to quantify the statistical association of data points considering multiple variables and generates a sub-sampled data that preserves the statistical association among multi-variables. Using such reduced sampled data, we show that multivariate feature query and analysis can be done effectively. The efficacy of the proposed multivariate association driven sampling algorithm is presented by applying it on several scientific data sets.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multivariate Pointwise Information-Driven Data Sampling and Visualization

Dutta

Biswas

Ahrens

2019

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…In situ analysis algorithms may transform data into reduced representations or surrogate models in order to mitigate large data size, high dimensionality, or long computation times. Low-rank approximation (Austin et al, 2016), statistical summarization (Biswas et al, 2018;Dutta et al, 2017;Hazarika et al, 2018;Lawrence et al, 2017;Lohrmann et al, 2017;Thompson et al, 2011), topological segmentation (Gyulassy et al, 2012(Gyulassy et al, , 2019Landge et al, 2014;Weber, 2013, 2014), wavelet transformation (Li et al, 2017;Salloum et al, 2018), lossy compression (Brislawn et al, 2012;Di and Cappello, 2016;Lindstrom, 2014), geometric modeling (Nashed et al, 2019; Peterka et al, 2018), and feature detection (Guo et al, 2017) may be used to generate reduced or surrogate models.…”

Section: Analysis Algorithmsmentioning

confidence: 99%

“…Research is required to modify existing post hoc algorithms and develop new in situ algorithms to satisfy the needs of modern use cases on emerging system architectures that can feature massive scale, many cores, deep memory hierarchies, or embedded lightweight edge devices. Examples of such algorithms include reduced representations and low-rank approximations (Austin et al, 2016), statistical (Biswas et al, 2018;Dutta et al, 2017;Hazarika et al, 2018;Thompson et al, 2011), topological (Gyulassy et al, 2012(Gyulassy et al, , 2019Landge et al, 2014;Weber, 2013, 2014), wavelets (Li et al, 2017;Salloum et al, 2018), compression (Brislawn et al, 2012;Di and Cappello, 2016;Lindstrom, 2014), and feature detection (Guo et al, 2017) methods. Surrogate models and multifidelity models can be geometric (Nashed et al, 2019;Peterka et al, 2018), statistical (Lawrence et al, 2017;Lohrmann et al, 2017), or neural network (He et al, 2019).…”

Section: In Situ Algorithmsmentioning

confidence: 99%

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

Peterka

Bard

Bennett

et al. 2020

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

show abstract

“…To deal with large amounts of data, recent approaches employ probabilistic data summaries [7,8,12,45] to represent blocks of data as probability distributions. These approaches have been mostly limited to univariate, volumetric data.…”

Section: Introductionmentioning

confidence: 99%

Void-and-Cluster Sampling of Large Scattered Data and Trajectories

Rapp

Peters

Dachsbacher

2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

We propose a data reduction technique for scattered data based on statistical sampling. Our void-and-cluster sampling technique finds a representative subset that is optimally distributed in the spatial domain with respect to the blue noise property. In addition, it can adapt to a given density function, which we use to sample regions of high complexity in the multivariate value domain more densely. Moreover, our sampling technique implicitly defines an ordering on the samples that enables progressive data loading and a continuous level-of-detail representation. We extend our technique to sample time-dependent trajectories, for example pathlines in a time interval, using an efficient and iterative approach. Furthermore, we introduce a local and continuous error measure to quantify how well a set of samples represents the original dataset. We apply this error measure during sampling to guide the number of samples that are taken. Finally, we use this error measure and other quantities to evaluate the quality, performance, and scalability of our algorithm.Data reduction is thus a necessary means to reduce storage requirements for both simulation, measurement devices, and for subsequent data analysis.We specifically consider the reduction of large, spatio-temporal scattered data, i.e. unstructured points in space-time with an associated value domain. In particular, we investigate the use of statistical sampling to reduce large data sets to a representative subset. Sampling scales well to higher dimensional data and is well-suited for scattered data. Although simple random sampling gives decent results, recent work improves upon this using stratified [21, 28] and informationguided sampling [3,27]. These results emphasize the significance of 1 arXiv:1907.05073v1 [cs.GR]

show abstract

Homogeneity guided probabilistic data summaries for analysis and visualization of large-scale data sets

Cited by 25 publications

References 34 publications

Multivariate Pointwise Information-Driven Data Sampling and Visualization

Multivariate Pointwise Information-Driven Data Sampling and Visualization

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

Void-and-Cluster Sampling of Large Scattered Data and Trajectories

Contact Info

Product

Resources

About