Overview of accurate coresets

Abstract

A coreset of an input set is its small summarization, such that solving a problem on the coreset as its input, provably yields the same result as solving the same problem on the original (full) set, for a given family of problems (models/classifiers/loss functions). Coresets have been suggested for many fundamental problems, for example, in machine/deep learning, computer vision, databases, and theoretical computer science. This introductory paper was written following requests regarding the many inconsistent coreset definitions, lack of source code, the required deep theoretical background from different fields, and the dense papers that make it hard for beginners to apply and develop coresets. The article provides folklore, classic, and simple results including step‐by‐step proofs and figures, for the simplest (accurate) coresets. Nevertheless, we did not find most of their constructions in the literature. Moreover, we expect that putting them together in a retrospective context would help the reader to grasp current results that usually generalize these fundamental observations. Experts might appreciate the unified notation and comparison table for existing results. Open source code is provided for all presented algorithms, to demonstrate their usage, and to support the readers who are more familiar with programming than mathematics. This article is categorized under: Algorithmic Development > Structure Discovery Fundamental Concepts of Data and Knowledge > Big Data Mining Technologies > Machine Learning

Provable Data Subset Selection For Efficient Neural Network Training

“…The tuple (P, w, X, f ) is called the query space, and it defines the optimization problem at hand -where usually, the goal is to find x * ∈ arg min x∈X p∈P w(p)f (p, x). Given a query space (P, w, X, f ), a coreset is a small weighted subset of the input P that can provably approximate the cost of every query x ∈ X on P Jubran et al, 2021); see Definition 2.1. In particular, a coreset for a RBFNN can approximate the cost of an RBFNN on the original training data for every set of centers and weights that define the RBFNN (see Section 4).…”