An efficient implementation of kernel density estimation for multi-core and many-core architectures

Abstract

Kernel density estimation (KDE) is a statistical technique used to estimate the probability density function of a sample set with unknown density function. It is considered a fundamental data-smoothing problem for use with large datasets, and is widely applied in areas such as climatology and biometry. Due to the large volumes of data that these problems usually process, KDE is a computationally challenging problem. Current HPC platforms with built-in accelerators have an enormous computing power, but they have to be programmed efficiently in order to take advantage of that power. We have developed a novel strategy to compute KDE using bounded kernels, trying to minimize memory accesses, and implemented it as a parallel program targeting multi-core and many-core processors. The efficiency of our code has been tested with different datasets, obtaining impressive levels of acceleration when taking as reference alternative, state-of-the-art KDE implementations.

On Mautner-Type Probability of Capture of Intergalactic Meteor Particles by Habitable Exoplanets

“…We defined the separation between grid points, so that we have 800 points in each dimension of 2D parameter space defined by exoplanet distances and radii. The output is KDE of the same dimensions as defined grid [37,38].…”

Probabilistic modeling of renewable energy source based on Spark platform with large-scale sample data

“…In recent decades, approximation‐based techniques and parallel approaches were widely used to reduce the computational time of probabilistic modeling. The former including the approximation of fast Fourier transforms and fast Gauss transform deteriorate the estimation accuracy of probability models at some degree.…”

Space-Time Kernel Density Estimation for Real-Time Interactive Visual Analytics

“…Calculating the temporal and spatial bandwidth is not a defined science either. In this work we used the approach by Silverman et al [29] as it was used by Lopez-Novoa et al [21]. Since we compared our work to Lopez-Novoa we used the same approach in order to maintain consistency.…”

“…In the first part of our performance evaluation we examine our techniques against the state of the art [21]. In their work they evaluate a 3D KDE by generating two synthetic datasets with the mvrnorm function from the R framework, which generates multivariate samples with a normal distribution.…”

“…Michailidis et al [24] focused on using the GPU to accelerate the KDE using memory sharing techniques. The most recent work in this area by LopezNovoa et al [21] used crop and chop techniques to accelerate the KDE. As part of their work they reviewed all prior KDE techniques and benchmarked performance.…”