Computing Spatial Distance Histograms for Large Scientific Data Sets On-the-Fly

Abstract

This paper focuses on an important query in scientific simulation data analysis: the Spatial Distance Histogram (SDH). The computation time of an SDH query using brute force method is quadratic. Often, such queries are executed continuously over certain time periods, increasing the computation time. We propose highly efficient approximate algorithm to compute SDH over consecutive time periods with provable error bounds. The key idea of our algorithm is to derive statistical distribution of distances from the spatial and temporal characteristics of particles. Upon organizing the data into a Quad-tree based structure, the spatiotemporal characteristics of particles in each node of the tree are acquired to determine the particles’ spatial distribution as well as their temporal locality in consecutive time periods. We report our efforts in implementing and optimizing the above algorithm in Graphics Processing Units (GPUs) as means to further improve the efficiency. The accuracy and efficiency of the proposed algorithm is backed by mathematical analysis and results of extensive experiments using data generated from real simulation studies.

DCMS: A data analytics and management system for molecular simulation

“…More efforts put into developing a Shared Memory (SM) based program yielded amazing returns of a 258 × improvement. The GPU version of the algorithm using spatial locality was also experimented, and reported a 20 × improvement (see [ 25 ] for more details). Based on our findings we believe that the GPUs are very powerful in processing complex analytics related to MS.…”

“…The entire distribution (i.e., histogram) can thus be obtained by considering all such pairs. Similarly, the locality of atoms in different frames is also utilized to compute approximate SDH very efficiently [ 25 ].…”

“…This makes code optimization in GPUs a very challenging task. We developed and optimized GPU versions of above SDH algorithms and achieved dramatic speedup of computation [ 25 ].…”

A Comparative Study of Dual-Tree Algorithms for Computing Spatial Distance Histograms

“…On top of that, we develop new models to compare the two data structures of interest in this paper. Our recent work on this topic [26], [27], [28] focuses on approximate SDH processing and parallel computing. Such work, again, only considers the quad/oct-tree as the underlying data structure thus has little overlap with this paper.…”