Near-Optimal Coresets of Kernel Density Estimates

Phillips, Jeff M.; Tai, Wai Ming

doi:10.1007/s00454-019-00134-6

Cited by 27 publications

(28 citation statements)

References 41 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sketch matrices may support the stronger turn‐style model that allows deletion and changing single entries and not only insertion of records, in sub‐linear space and without using trees. See surveys in Clarkson and Woodruff () and Phillips ().…”

Section: Coreset Typesmentioning

confidence: 99%

See 1 more Smart Citation

Core‐sets: An updated survey

Feldman

2019

WIREs Data Min & Knowl

View full text Add to dashboard Cite

In optimization or machine learning problems we are given a set of items, usually points in some metric space, and the goal is to minimize or maximize an objective function over some space of candidate solutions. For example, in clustering problems, the input is a set of points in some metric space, and a common goal is to compute a set of centers in some other space (points, lines) that will minimize the sum of distances to these points. In database queries, we may need to compute such a some for a specific query set of k centers. However, traditional algorithms cannot handle modern systems that require parallel real‐time computations of infinite distributed streams from sensors such as GPS, audio or video that arrive to a cloud, or networks of weaker devices such as smartphones or robots. Core‐set is a “small data” summarization of the input “big data,” where every possible query has approximately the same answer on both data sets. Generic techniques enable efficient coreset maintenance of streaming, distributed and dynamic data. Traditional algorithms can then be applied on these coresets to maintain the approximated optimal solutions. The challenge is to design coresets with provable tradeoff between their size and approximation error. This survey summarizes such constructions in a retrospective way, that aims to unified and simplify the state‐of‐the‐art. This article is categorized under Algorithmic Development > Structure Discovery Fundamental Concepts of Data and Knowledge > Big Data Mining Technologies > Machine Learning Algorithmic Development > Scalable Statistical Methods

show abstract

Section: Coreset Typesmentioning

confidence: 99%

“…They are thus composable and it is usually easy to extract from them an approximation to the optimal query of the original input P . See examples in P. Agarwal et al (), Phillips (), and Feldman and Langberg ().…”

Section: Coreset Typesmentioning

confidence: 99%

Core‐sets: An updated survey

Feldman

2019

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…Kernel Density. In the context of Kernel Density Estimation [29], Coresets [41,55] have received renewed attention in recent years resulting in near optimal constructions [57] for certain cases. The literature has mostly been focused on obtaining additive error ε > 0.…”

Section: Core-setsmentioning

confidence: 99%

“…Gaussian, Laplacian) using a greedy construction (kernel herding [25]). The other approach [55,56,57] applies to Lipschitz kernels of bounded influence (decay fast enough), and constructs the core-set by starting with the full set of points and reducing it by half each time. Using smoothness properties of the kernel one can then bound the error introduced by each such operation through the notion of discrepancy [24,23].…”

Section: Core-setsmentioning

confidence: 99%

Multi-resolution Hashing for Fast Pairwise Summations

Charikar

Siminelakis

2019

2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS)

View full text Add to dashboard Cite

A basic computational primitive in the analysis of massive datasets is summing simple functions over a large number of objects. Modern applications pose an additional challenge in that such functions often depend on a parameter vector y (query) that is unknown a priori. Given a set of points X ⊂ d and a pairwise function w : d × d → [0, 1], we study the problem of designing a data-structure that enables sublinear-time approximation of the summation Z w (y) 1 |X| x∈X w(x, y) for any query y ∈ d . By combining ideas from Harmonic Analysis (partitions of unity and approximation theory) with Hashing-Based-Estimators [Charikar, Siminelakis FOCS'17], we provide a general framework for designing such data structures through hashing that reaches far beyond what previous techniques allowed.A key design principle is a collection of T 1 hashing schemes with collision probabilities p 1 , . . . , p T such that sup t∈[T] {p t (x, y)} Θ( w(x, y)). This leads to a data-structure that approximates Z w (y) using a sub-linear number of samples from each hash family. Using this new framework along with Distance Sensitive Hashing [Aumuller, Christiani, Pagh, Silvestri PODS'18], we show that such a collection can be constructed and evaluated efficiently for any log-convex function w(x, y) e φ( x, y ) of the inner product on the unit sphere x, y ∈ S d−1 .Our method leads to data structures with sub-linear query time that significantly improve upon random sampling and can be used for Kernel Density or Partition Function Estimation. We provide extensions of our result from the sphere to d and from scalar functions to vector functions.

show abstract

“…Instead of processing the original dataset, one can perform the computation on its coreset with little loss of accuracy. Various types of problems have been shown to be effective under coreset approximation, e.g., kmedian and k-means clustering [3,9,20], non-negative matrix factorization (NMF) [17], kernel density estimation (KDE) [26,34], and many others [6,10,21].…”

Section: Introductionmentioning

confidence: 99%

Coresets for Minimum Enclosing Balls over Sliding Windows

Wang

Tan

2019

Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Coresets are important tools to generate concise summaries of massive datasets for approximate analysis. A coreset is a small subset of points extracted from the original point set such that certain geometric properties are preserved with provable guarantees. This paper investigates the problem of maintaining a coreset to preserve the minimum enclosing ball (MEB) for a sliding window of points that are continuously updated in a data stream. Although the problem has been extensively studied in batch and append-only streaming settings, no efficient sliding-window solution is available yet. In this work, we first introduce an algorithm, called AOMEB, to build a coreset for MEB in an append-only stream. AOMEB improves the practical performance of the state-of-the-art algorithm while having the same approximation ratio. Furthermore, using AOMEB as a building block, we propose two novel algorithms, namely SWMEB and SWMEB+, to maintain coresets for MEB over the sliding window with constant approximation ratios. The proposed algorithms also support coresets for MEB in a reproducing kernel Hilbert space (RKHS). Finally, extensive experiments on real-world and synthetic datasets demonstrate that SWMEB and SWMEB+ achieve speedups of up to four orders of magnitude over the state-of-the-art batch algorithm while providing coresets for MEB with rather small errors compared to the optimal ones.

show abstract

Near-Optimal Coresets of Kernel Density Estimates

Cited by 27 publications

References 41 publications

Core‐sets: An updated survey

Core‐sets: An updated survey

Multi-resolution Hashing for Fast Pairwise Summations

Coresets for Minimum Enclosing Balls over Sliding Windows

Contact Info

Product

Resources

About