High performance multivariate visual data exploration for extremely large data

Rübel, Oliver; Prabhat,; Wu, Kesheng; Childs, Hank; Meredith, Jeremy; Geddes, C. G. R.; Cormier‐Michel, E.; Ahern, Sean; Weber, Gunther H.; Messmer, Peter; Hagen, Hans; Hamann, Bernd; Bethel, E. Wes

doi:10.1109/sc.2008.5214436

Cited by 42 publications

(74 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Laser wakefield simulations model the behavior of individual particles, and the electric and magnetic fields associated with the plasma. Scientists at the LBNL LOASIS facility are currently using the VORPAL program to model physical experiments, gain deeper understanding of the physical observations and optimize methodology for future experiments [8,22].…”

Section: Laser Wakefield Particle Acceleratormentioning

confidence: 99%

FastQuery: A Parallel Indexing System for Scientific Data

Chou

Prabhat

2011

2011 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

Modern scientific datasets present numerous data management and analysis challenges. State-of-theart index and query technologies such as FastBit can significantly improve accesses to these datasets by augmenting the user data with indexes and other secondary information. However, a challenge is that the indexes assume the relational data model but the scientific data generally follows the array data model. To match the two data models, we design a generic mapping mechanism and implement an efficient input and output interface for reading and writing the data and their corresponding indexes. To take advantage of the emerging many-core architectures, we also develop a parallel strategy for indexing using threading technology. This approach complements our on-going MPI-based parallelization efforts.We demonstrate the flexibility of our software by applying it to two of the most commonly used scientific data formats, HDF5 and NetCDF. We present two case studies using data from a particle accelerator model and a global climate model. We also conducted a detailed performance study using these scientific datasets. The results show that FastQuery speeds up the query time by a factor of 2.5x to 50x, and it reduces the indexing time by a factor of 16 on 24 cores.

show abstract

Section: Laser Wakefield Particle Acceleratormentioning

confidence: 99%

FastQuery: A Parallel Indexing System for Scientific Data

Chou

Prabhat

2011

2011 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

show abstract

“…FastBit provides a mechanism to directly input the list of IDs to reduce the query response time. This feature was also used in an earlier work by Rübel et al [7].…”

Section: High Performance Index/query For Data Miningmentioning

confidence: 99%

“…FastBit has shown to be effective for computing 1D and 2D conditional histograms [41,7]. In this work, we extend FastBit to enable fast computation of 3D conditional histograms and to export bit vectors for representing particles (data records) associated with the bins of a histogram.…”

Section: High Performance Index/query For Data Miningmentioning

confidence: 99%

“…First, VisIt provides direct integration of FastBit. Second, VisIt supports selection of particles based on particle IDs via the concept of named selections [7]. Once we define which particles form a bunch, this capability allows us to save the IDs of the selected particles in the form of a named selection, and then apply this selection directly to the original data.…”

Section: High Performance Index/query For Data Miningmentioning

confidence: 99%

“…These processes are best understood by tracing the particles that form a beam over time and investigating their temporal evolution [5,6,7,8]. In real-world experiments it is, however, impossible to record the complete evolution of an experiment and much less to trace single particles within a plasma.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic beam path analysis of laser wakefield particle acceleration data

Rübel¹,

Geddes²,

Cormier‐Michel³

et al. 2009

Comput. Sci. Disc.

Self Cite

View full text Add to dashboard Cite

Abstract.Numerical simulations of laser wakefield particle accelerators play a key role in the understanding of the complex acceleration process and in the design of expensive experimental facilities. As the size and complexity of simulation output grows, an increasingly acute challenge is the practical need for computational techniques that aid in scientific knowledge discovery. To that end, we present a set of data-understanding algorithms that work in concert in a pipeline fashion to automatically locate and analyze high energy particle bunches undergoing acceleration in very large simulation datasets. These techniques work cooperatively by first identifying features of interest in individual timesteps, then integrating features across timesteps, and based on the information derived perform analysis of temporally dynamic features. This combination of techniques supports accurate detection of particle beams enabling a deeper level of scientific understanding of physical phenomena than has been possible before. By combining efficient data analysis algorithms and state-of-the-art data management we enable high-performance analysis of extremely large particle datasets in 3D. We demonstrate the usefulness of our methods for a variety of 2D and 3D datasets and discuss the performance of our analysis pipeline.

show abstract