We describe Chromium, a system for manipulating streams of graphics API commands on clusters of workstations. Chromium's stream filters can be arranged to create sort-first and sort-last parallel graphics architectures that, in many cases, support the same applications while using only commodity graphics accelerators. In addition, these stream filters can be extended programmatically, allowing the user to customize the stream transformations performed by nodes in a cluster. Because our stream processing mechanism is completely general, any cluster-parallel rendering algorithm can be either implemented on top of or embedded in Chromium. In this paper, we give examples of real-world applications that use Chromium to achieve good scalability on clusters of workstations, and describe other potential uses of this stream processing technology. By completely abstracting the underlying graphics architecture, network topology, and API command processing semantics, we allow a variety of applications to run in different environments.
As scientific instruments and computer simulations produce more and more data, the task of locating the essential information to gain insight becomes increasingly difficult. FastBit is an efficient software tool to address this challenge. In this article, we present a summary of the key techniques, namely bitmap compression, encoding and binning. The advances in these techniques have led to a search tool that can answer structured (SQL) queries orders of magnitude faster than popular database systems. To illustrate how FastBit is used in applications, we present three examples involving a high-energy physics experiment, a combustion simulation, and an accelerator simulation. In each case, FastBit significantly reduces the response time and enables interactive exploration on terabytes of data.
One of the central challenges in modern science is the need to quickly derive knowledge and understanding from large, complex collections of data. We present a new approach that deals with this challenge by combining and extending techniques from high performance visual data analysis and scientific data management. This approach is demonstrated within the context of gaining insight from complex, time-varying datasets produced by a laser wakefield accelerator simulation. Our approach leverages histogram-based parallel coordinates for both visual information display as well as a vehicle for guiding a data mining operation. Data extraction and subsetting are implemented with state-of-the-art index/query technology. This approach, while applied here to accelerator science, is generally applicable to a broad set of science applications, and is implemented in a production-quality visual data analysis infrastructure. We conduct a detailed performance analysis and demonstrate good scalability on a distributed memory Cray XT4 system.
Understanding vector fields resulting from large scientific simulations is an important and often difficult task. Streamlines, curves that are tangential to a vector field at each point, are a powerful visualization method in this context. Application of streamline-based visualization to very large vector field data represents a significant challenge due to the non-local and data-dependent nature of streamline computation, and requires careful balancing of computational demands placed on I/O, memory, communication, and processors. In this paper we review two parallelization approaches based on established parallelization paradigms (static decomposition and on-demand loading) and present a novel hybrid algorithm for computing streamlines. Our algorithm is aimed at good scalability and performance across the widely varying computational characteristics of streamlinebased problems. We perform performance and scalability studies of all three algorithms on a number of prototypical application problems and demonstrate that our hybrid scheme is able to perform well in different settings.
Magnetically confined plasmas can contain significant concentrations of nonthermal plasma particles arising from fusion reactions, neutral beam injection, and wave-driven diffusion in velocity space. Initial studies in one-dimensional and experimental results show that nonthermal energetic ions can significantly affect wave propagation and heating in the ion cyclotron range of frequencies. In addition, these ions can absorb power at high harmonics of the cyclotron frequency where conventional two-dimensional global-wave models are not valid. In this work, the all-orders global-wave solver AORSA [E. F. Jaeger et al., Phys. Rev. Lett. 90, 195001 (2003)] is generalized to treat non-Maxwellian velocity distributions. Quasilinear diffusion coefficients are derived directly from the wave fields and used to calculate energetic ion velocity distributions with the CQL3D Fokker-Planck code [R. W. Harvey and M. G. McCoy, Proceedings of the IAEA Technical Committee Meeting on Simulation and Modeling of Thermonuclear Plasmas, Montreal, Canada, 1992 (USDOC NTIS Document No. DE93002962)]. For comparison, the quasilinear coefficients can be calculated numerically by integrating the Lorentz force equations along particle orbits. Self-consistency between the wave electric field and resonant ion distribution function is achieved by iterating between the global-wave and Fokker-Planck solutions.
The next step toward fusion as a practical energy source is the design and construction of ITER [R. Aymar et al., Nucl. Fusion 41, 1301 (2001)], a device capable of producing and controlling the high-performance plasma required for self-sustaining fusion reactions, i.e., “burning plasma.” ITER relies in part on ion-cyclotron radio frequency power to heat the deuterium and tritium fuel to fusion temperatures. In order to heat effectively, the radio frequency wave fields must couple efficiently to the dense core plasma. Calculations in this paper support the argument that this will be the case. Three-dimensional full-wave simulations show that fast magnetosonic waves in ITER propagate radially inward with strong central focusing and little toroidal spreading. Energy deposition, current drive, and plasma flow are all highly localized near the plasma center. Very high resolution, two-dimensional calculations reveal the presence of mode conversion layers, where fast waves can be converted to slow ion cyclotron waves. When minority ions such as deuterium or helium-3 are used to damp the launched waves, these ions can be accelerated to high energies, forming suprathermal tails that significantly affect the wave propagation and absorption. By neglecting the toroidal localization of the waves and the finite radial excursion of the energetic particle orbits, the quasilinear evolution of these suprathermal ion tails can be simulated self-consistently in one spatial dimension and two velocity dimensions.
There are some important and motivating questions that drive the research for processing massive data sets, like will it be possible to use the simpler pure parallelism technique to process tomorrow's data? Can pure parallelism scale sufficiently to process massive data sets?To answer these questions, the researchers performed a series of experiments, originally published in IEEE Computer Graphics and Applications [2] and forming the basis of this report, that studied the scalability of pure parallelism in visualization software on massive data sets. These experiments utilized multiple visualization algorithms and were run on multiple architectures. There were two types of experiments performed. The first experiment examined performance at a massive scale: 16,000 or more cores and one trillion or more cells. The second experiment studied whether the approach can maintain a fixed amount of time to complete an operation when the data size is doubled and the amount of resources is doubled, also known as weak scalability. At the time of their original publication, these experiments represented the largest data set sizes ever published in visualization literature. Further, their findings still continue to contribute to the understanding of today's dominant processing paradigm (pure parallelism) on tomorrow's data, in the form of scaling characteristics and bottlenecks at high levels of concurrency and with very large data sets.
We present the results of a series of experiments studying how visualization software scales to massive data sets. Although several paradigms exist for processing large data, we focus on pure parallelism, the dominant approach for production software. These experiments utilized multiple visualization algorithms and were run on multiple architectures. Two types of experiments were performed. For the first, we examined performance at massive scale: 16,000 or more cores and one trillion or more cells. For the second, we studied weak scaling performance. These experiments were performed on the largest data set sizes published to date in visualization literature, and the findings on scaling characteristics and bottlenecks contribute to understanding of how pure parallelism will perform at high levels of concurrency and with very large data sets.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.