Nearly Optimal Solutions for the Chow Parameters Problem and Low-Weight Approximation of Halfspaces

De, Anindya; Diakonikolas, Ilias; Feldman, Vitaly; Servedio, Rocco A.

doi:10.1145/2590772

Cited by 33 publications

(60 citation statements)

References 48 publications

(99 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…So Theorem 7 shows that for a specific natural function, taking k to be constant and letting ε vary, getting an ε-approximator over the Hamming ball {0, 1} n ≤k (for k constant) requires weights that are exponentially larger than the weights required for ε-approximation over the entire Boolean cube. Theorem 7 is also in sharp contrast with the recent upper bound of [2] which shows that there is always an ε-approximator over the entire Boolean cube which has weight at most quasipoly(1/ε) (as a function of ε).…”

Section: Lower Bounds For Approximat-ing Halfspacesmentioning

confidence: 68%

See 2 more Smart Citations

Low-weight halfspaces for sparse boolean vectors

Long¹,

Servedio

2013

Proceedings of the 4th Conference on Innovations in Theoretical Computer Science

Self Cite

View full text Add to dashboard Cite

For S ⊆ {0, 1}n , a Boolean function f : S → {−1, 1} is a halfspace over S if there exist w ∈ R n and θ ∈ R such that f (x) = sign(w · x − θ) for all x ∈ S. We give bounds on the size of integer weights w1, . . . , wn ∈ Z that are required to represent halfspaces over Hamming balls centered at 0 n , i.e. halfspaces over S = {0, 1}n ≤k def = {x ∈ {0, 1} n : x1 + · · · + xn ≤ k}. Such weight bounds for halfspaces over Hamming balls have immediate consequences for the performance of learning algorithms in the increasingly common scenario of learning from very high-dimensional categorical examples which are such that only a small number of features are active in each example.We give upper and lower bounds on weight both for exact representation (when sign(w · x − θ) must equal f (x) for every x ∈ S) and for ε-approximate representation (when sign(w · x − θ) may disagree with f (x) for up to an ε fraction of points x ∈ S). Our results show that extremal bounds for exact representation are qualitatively rather similar whether the domain is all of {0, 1} n or the Hamming ball {0, 1} n ≤k , but extremal bounds for approximate representation are qualitatively very different between these two domains.

show abstract

Section: Lower Bounds For Approximat-ing Halfspacesmentioning

confidence: 68%

“…(1/ε 2/3 ) by [3], and very recently [2] have further improved the upper bound to √ n · (1/ε) O(log 2 (1/ε)) .…”

Section: Prior Work On Approximation Over {0 1}mentioning

confidence: 99%

See 1 more Smart Citation

Low-weight halfspaces for sparse boolean vectors

Long¹,

Servedio

2013

Proceedings of the 4th Conference on Innovations in Theoretical Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…[Ser07] showed that every n-variable halfspace over {0, 1} n can be ε-approximated by a halfspace of weight √ n · 2Õ (1/ε 2 ) , and showed an Ω( √ n) lower bound for constant ε. The upper bound was subsequently improved (as a function of ε) to weight n 3/2 · 2Õ (1/ε 2/3 ) by [DS09], and very recently [DDFS12] have further improved the upper bound to √ n · (1/ε) O(log 2 (1/ε)) .…”

Section: Previous Work and Our Resultsmentioning

confidence: 99%

On the Weight of Halfspaces over Hamming Balls

Long

Servedio²

2014

SIAM J. Discrete Math.

Self Cite

View full text Add to dashboard Cite

For S ⊆ {0, 1}n , a Boolean function f : S → {−1, 1} is a halfspace over S if there exist w ∈ R n and θ ∈ R such that f (x) = sign(w · x − θ) for all x ∈ S. We give bounds on the size of integer weights w 1 , . . . , w n ∈ Z that are required to represent halfspaces over Hamming balls S = {x ∈ {0, 1} n : x 1 + · · · + x n ≤ k}. Such weight bounds for halfspaces over Hamming balls have immediate consequences for the performance of learning algorithms in the common scenario of learning from very high-dimensional categorical examples which are such that only a small number of features are active in each example.We give upper and lower bounds on weight both for exact representation (when sign(w · x−θ) must equal f (x) for every x ∈ S) and for ε-approximate representation (when sign(w · x−θ) may disagree with f (x) for up to an ε fraction of points x ∈ S). Our results show that extremal bounds for exact representation are qualitatively rather similar whether the domain is all of {0, 1} n or the Hamming ball {0, 1} n ≤k , but extremal bounds for approximate representation are qualitatively very different between these two domains.

show abstract

“…The high level idea in the proof of Theorem 9 is to exploit the so-called "structure versus randomness" phenomenon for halfspaces which was introduced in the influential work of Servedio [43] and has subsequently played a crucial role in the recent developments in the complexity theoretic analysis of halfspaces [43,18,15,33,32] (we explain this phenomenon a little later). Results in this line of work have mostly looked at halfspaces of the form g(X 1 , .…”

Section: Theoremmentioning

confidence: 99%

Boolean function analysis meets stochastic optimization: An approximation scheme for stochastic knapsack

De¹

2018

Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms

Self Cite

View full text Add to dashboard Cite

The stochastic knapsack problem is the stochastic variant of the classical knapsack problem in which the algorithm designer is given a a knapsack with a given capacity and a collection of items where each item is associated with a profit and a probability distribution on its size. The goal is to select a subset of items with maximum profit and violate the capacity constraint with probability at most p (referred to as the overflow probability).While several approximation algorithms [27,22,4,17,30] have been developed for this problem, most of these algorithms relax the capacity constraint of the knapsack. In this paper, we design efficient approximation schemes for this problem without relaxing the capacity constraint.(i) Our first result is in the case when item sizes are Bernoulli random variables. In this case, we design a (nearly) fully polynomial time approximation scheme (FPTAS) which only relaxes the overflow probability.(ii) Our second result generalizes the first result to the case when all the item sizes are supported on a (common) set of constant size. In this case, we obtain a quasi-FPTAS.(iii) Our third result is in the case when item sizes are socalled "hypercontractive" random variables i.e., random variables whose second and fourth moments are within constant factors of each other. In other words, the kurtosis of the random variable is upper bounded by a constant. This class has been widely studied in probability theory and most natural random variables are hypercontractive including well-known families such as Poisson, Gaussian, exponential and Laplace distributions. In this case, we design a polynomial time approximation scheme which relaxes both the overflow probability and maximum profit.Crucially, all of our algorithms meet the capacity constraint exactly, a result which was previously known only when the item sizes were Poisson or Gaussian random variables [22,24]. Our results rely on new connections between Boolean function analysis and stochastic optimization and are obtained by an adaption and extension of ideas such as (central) limit theorems, moment matching theorems and the influential critical index machinery of Servedio [43] developed in the context of complexity theoretic analysis of halfspaces. We believe that these ideas and techniques may prove to be useful in other stochastic optimization problems as well.

show abstract

Nearly Optimal Solutions for the Chow Parameters Problem and Low-Weight Approximation of Halfspaces

Cited by 33 publications

References 48 publications

Low-weight halfspaces for sparse boolean vectors

Low-weight halfspaces for sparse boolean vectors

On the Weight of Halfspaces over Hamming Balls

Boolean function analysis meets stochastic optimization: An approximation scheme for stochastic knapsack

Contact Info

Product

Resources

About