Polynomial regression under arbitrary product distributions

Blais, Eric; O’Donnell, Ryan; Wimmer, Karl

doi:10.1007/s10994-010-5179-6

Cited by 31 publications

(39 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Efron-Stein decomposition method has found numerous applications in statistics [8,13,19], hardness of approximation [1,14,15], learning theory [4], and social choice theory [15]. As we see below, the method is also particularly well suited for the analysis of juntas.…”

Section: Our Techniquesmentioning

confidence: 99%

Testing juntas nearly optimally

Blais

2009

Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing

Self Cite

150

View full text Add to dashboard Cite

A function on n variables is called a k-junta if it depends on at most k of its variables. In this article, we show that it is possible to test whether a function is a k-junta or is "far" from being a k-junta with O(k/ + k log k) queries, where is the approximation parameter. This result improves on the previous best upper bound ofÕ(k 3/2 )/ queries and is asymptotically optimal, up to a logarithmic factor.We obtain the improved upper bound by introducing a new algorithm with one-sided error for testing juntas. Notably, the algorithm is a valid junta tester under very general conditions: it holds for functions with arbitrary finite domains and ranges, and it holds under any product distribution over the domain.A key component of the analysis of the new algorithm is a new structural result on juntas: roughly, we show that if a function f is "far" from being a k-junta, then f is "far" from being determined by k parts in a random partition of the variables. The structural lemma is proved using the Efron-Stein decomposition method.

show abstract

Section: Our Techniquesmentioning

confidence: 99%

Testing juntas nearly optimally

Blais

2009

Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing

Self Cite

150

View full text Add to dashboard Cite

show abstract

“…[22,26,3,15,23,39]. Let DX be a (fixed, known) distribution over an example space X such as the uniform distribution over {−1, 1} n or the standard multivariate Gaussian distribution N (0, In) over R n .…”

Section: Application: Agnostically Learning Constantdegree Ptfs In Pomentioning

confidence: 99%

Bounding the average sensitivity and noise sensitivity of polynomial threshold functions

Diakonikolas

Harsha

Klivans

et al. 2010

Proceedings of the Forty-Second ACM Symposium on Theory of Computing

View full text Add to dashboard Cite

We give the first non-trivial upper bounds on the average sensitivity and noise sensitivity of degree-d polynomial threshold functions (PTFs). These bounds hold both for PTFs over the Boolean hypercube {−1, 1} n and for PTFs over R n under the standard n-dimensional Gaussian distribution N (0, In). Our bound on the Boolean average sensitivity of PTFs represents progress towards the resolution of a conjecture of Gotsman and Linial [17], which states that the symmetric function slicing the middle d layers of the Boolean hypercube has the highest average sensitivity of all degree-d PTFs. Via the L1 polynomial regression algorithm of Kalai et al. [22], our bounds on Gaussian and Boolean noise sensitivity yield polynomial-time agnostic learning algorithms for the broad class of constant-degree PTFs under these input distributions.The main ingredients used to obtain our bounds on both average and noise sensitivity of PTFs in the Gaussian setting are tail bounds and anti-concentration bounds on low-degree polynomials in Gaussian random variables [20,7]. To obtain our bound on the Boolean average sensitivity of PTFs, we generalize the "critical-index" machinery of [37] (which in that work applies to halfspaces, i.e. degree-1 PTFs) to general PTFs. Together with the "invariance principle" of [30], this lets us extend our techniques from the Gaussian setting to the Boolean setting. Our bound on Boolean noise sensitivity is achieved via a simple reduction from upper bounds on average sensitivity of Boolean PTFs to corresponding bounds on noise sensitivity.

show abstract

“…The "polynomial kernel" is a popular kernel to use in this way; when, as is usually the case, the degree parameter in the polynomial kernel is set to be a small constant, these algorithms output hypotheses that are equivalent to low-degree PTFs. Low-degree PTFs are also used as hypothe-ses in several important learning algorithms with a more complexity-theoretic flavor, such as the lowdegree algorithm of Linial et al [21] and its variants [12,22], including some algorithms for distributionspecific agnostic learning [14,20,3,6].…”

mentioning

confidence: 99%

Hardness Results for Agnostically Learning Low-Degree Polynomial Threshold Functions

Diakonikolas¹,

O’Donnell²,

Servedio³

et al. 2011

Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

Hardness results for maximum agreement problems have close connections to hardness results for proper learning in computational learning theory. In this paper we prove two hardness results for the problem of finding a low degree polynomial threshold function (PTF) which has the maximum possible agreement with a given set of labeled examples in R n × {−1, 1}. We prove that for any constants d 1, > 0,• Assuming the Unique Games Conjecture, no polynomial-time algorithm can find a degree-d PTF that is consistent with a ( • It is NP-hard to find a degree-2 PTF that is consistent with a ( 1 2 + ) fraction of a given set of labeled examples in R n × {−1, 1}, even if there exists a halfspace (degree-1 PTF) that is consistent with a 1 − fraction of the examples.These results immediately imply the following hardness of learning results: (i) Assuming the Unique Games Conjecture, there is no better-than-trivial proper learning algorithm that agnostically learns degree-d PTFs under arbitrary distributions; (ii) There is no better-than-trivial learning algorithm that outputs degree-2 PTFs and agnostically learns halfspaces (i.e. degree-1 PTFs) under arbitrary distributions.

show abstract

Polynomial regression under arbitrary product distributions

Cited by 31 publications

References 22 publications

Testing juntas nearly optimally

Testing juntas nearly optimally

Bounding the average sensitivity and noise sensitivity of polynomial threshold functions

Hardness Results for Agnostically Learning Low-Degree Polynomial Threshold Functions

Contact Info

Product

Resources

About