Density Estimation with Adaptive Sparse Grids for Large Data Sets

Peherstorfer, Benjamin; Pflüger, Dirk; Bungartz, H.-J.

doi:10.1137/1.9781611973440.51

Cited by 28 publications

(24 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It could be applied to other sparse grid algorithms used in, e.g., data-driven problems. There the optimization problems can often be related to elliptic PDEs and thus could be discretized with the auxiliary coefficients too [24,28].…”

Section: Discussionmentioning

confidence: 99%

A Multigrid Method for Adaptive Sparse Grids

Peherstorfer¹,

Zimmer

Zenger

et al. 2015

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

Abstract. Sparse grids have become an important tool to reduce the number of degrees of freedom of discretizations of moderately high-dimensional partial differential equations; however, the reduction in degrees of freedom comes at the cost of an almost dense and unconventionally structured system of linear equations. To guarantee overall efficiency of the sparse grid approach, special linear solvers are required. We present a multigrid method that exploits the sparse grid structure to achieve an optimal runtime that scales linearly with the number of sparse grid points. Our approach is based on a novel decomposition of the right-hand sides of the coarse grid equations that leads to a reformulation in so-called auxiliary coefficients. With these auxiliary coefficients, the right-hand sides can be represented in a nodal point basis on low-dimensional full grids. Our proposed multigrid method directly operates in this auxiliary coefficient representation, circumventing most of the computationally cumbersome sparse grid structure. Numerical results on nonadaptive and spatially adaptive sparse grids confirm that the runtime of our method scales linearly with the number of sparse grid points and they indicate that the obtained convergence factors are bounded independently of the mesh width. 1. Introduction. Many problems in computational science and engineering, e.g., in financial engineering, quantum mechanics, and data-driven prediction, lead to high-dimensional function approximations [23]. We consider here the case where the function is not given explicitly at selected sample points but implicitly as the solution of a high-dimensional partial differential equation (PDE). Standard discretizations on isotropic uniform grids, so-called full grids, become computationally infeasible because the number of degrees of freedom grows exponentially with the dimension. Sparse grid discretizations are a versatile way to circumvent this exponential growth [4]. If the function satisfies certain smoothness assumptions [4, section 3.1] sparse grids reduce the number of degrees of freedom to depend only logarithmically on the dimension; however, the discretization of a PDE on a sparse grid with the (piecewise linear) hierarchical basis results in an almost dense and unconventionally structured system of linear equations. To be computationally efficient solvers must take this structure into account. We develop a multigrid method that exploits the structure of sparse grid discretizations to achieve an optimal runtime that scales linearly with the number of sparse grids points.Multigrid methods have been extensively studied in the context of sparse grids. They are used either as preconditioners [1,11,12,15,16] or within real multigrid schemes [2,3,6,26,33]. We first note that certain problems allow for sparse grid dis-

show abstract

Section: Discussionmentioning

confidence: 99%

A Multigrid Method for Adaptive Sparse Grids

Peherstorfer¹,

Zimmer

Zenger

et al. 2015

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For a more detailed discussion of the achievable level of quality, we refer to prior work which compared sparse grid clustering to other clustering algorithms [21]. A comparison of the sparse grid density estimation to other density estimation methods is available as well [30].…”

Section: Clustering Quality and Parameter Tuningmentioning

confidence: 99%

Heterogeneous Distributed Big Data Clustering on Sparse Grids

2019

View full text Add to dashboard Cite

Clustering is an important task in data mining that has become more challenging due to the ever-increasing size of available datasets. To cope with these big data scenarios, a high-performance clustering approach is required. Sparse grid clustering is a density-based clustering method that uses a sparse grid density estimation as its central building block. The underlying density estimation approach enables the detection of clusters with non-convex shapes and without a predetermined number of clusters. In this work, we introduce a new distributed and performance-portable variant of the sparse grid clustering algorithm that is suited for big data settings. Our computed kernels were implemented in OpenCL to enable portability across a wide range of architectures. For distributed environments, we added a manager–worker scheme that was implemented using MPI. In experiments on two supercomputers, Piz Daint and Hazel Hen, with up to 100 million data points in a ten-dimensional dataset, we show the performance and scalability of our approach. The dataset with 100 million data points was clustered in 1198 s using 128 nodes of Piz Daint. This translates to an overall performance of 352 TFLOPS . On the node-level, we provide results for two GPUs, Nvidia’s Tesla P100 and the AMD FirePro W8100, and one processor-based platform that uses Intel Xeon E5-2680v3 processors. In these experiments, we achieved between 43% and 66% of the peak performance across all computed kernels and devices, demonstrating the performance portability of our approach.

show abstract

“…The PDF q of the biasing distribution is then constructed as a mixture model of normal distributions fitted to the samples in G r . Note that other density estimation techniques could be used instead as well [29,30,31]. It is reasonable to expect that q is a suitable biasing distribution for the failure domain G because if the surrogate model approximates the high-fidelity model well with respect to (13), the failure domain of the surrogate model G r and the failure domain G of the high-fidelity model have a large overlap.…”

Section: Step One: Constructing Biasing Distribution With the Surrogamentioning

confidence: 99%

Multifidelity importance sampling

Peherstorfer

Cui

Marzouk

et al. 2016

Computer Methods in Applied Mechanics and Engineering

Self Cite

114

View full text Add to dashboard Cite

Estimating statistics of model outputs with the Monte Carlo method often requires a large number of model evaluations. This leads to long runtimes if the model is expensive to evaluate. Importance sampling is one approach that can lead to a reduction in the number of model evaluations. Importance sampling uses a biasing distribution to sample the model more efficiently, but generating such a biasing distribution can be difficult and usually also requires model evaluations. A different strategy to speed up Monte Carlo sampling is to replace the computationally expensive high-fidelity model with a computationally cheap surrogate model; however, because the surrogate model outputs are only approximations of the high-fidelity model outputs, the estimate obtained using a surrogate model is in general biased with respect to the estimate obtained using the high-fidelity model. We introduce a multifidelity importance sampling (MFIS) method, which combines evaluations of both the high-fidelity and a surrogate model. It uses a surrogate model to facilitate the construction of the biasing distribution, but relies on a small number of evaluations of the high-fidelity model to derive an unbiased estimate of the statistics of interest. We prove that the MFIS estimate is unbiased even in the absence of accuracy guarantees on the surrogate model itself. The MFIS method can be used with any type of surrogate model, such as projection-based reduced-order models and data-fit models. Furthermore, the MFIS method is applicable to black-box models, i.e., where only inputs and the corresponding outputs of the high-fidelity and the surrogate model are available but not the details of the models themselves. We demonstrate on nonlinear and time-dependent problems that our MFIS method achieves speedups of up to several orders of magnitude compared to Monte Carlo with importance sampling that uses the high-fidelity model only.

show abstract

Density Estimation with Adaptive Sparse Grids for Large Data Sets

Cited by 28 publications

References 15 publications

A Multigrid Method for Adaptive Sparse Grids

A Multigrid Method for Adaptive Sparse Grids

Heterogeneous Distributed Big Data Clustering on Sparse Grids

Multifidelity importance sampling

Contact Info

Product

Resources

About