New Results for Learning Noisy Parities and Halfspaces

Feldman, Vitaly; Gopalan, Parikshit; Khot, Subhash; Ponnuswami, Ashok Kumar

doi:10.1109/focs.2006.51

Cited by 97 publications

(96 citation statements)

References 40 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus we may consider this equivalence as a new evidence of the hardness of attribute-efficient learning of parities from random examples only. This result together with a recent result of Feldman et al [FGKP06] implies equivalence of attribute-efficient learning of parities and learning of parities with random noise (for an appropriate transformation of parameters).…”

Section: Our Resultssupporting

confidence: 65%

“…FGKP06]. This allows us to relate the two learning problems directly, in particular, we obtain the following theorem.…”

Section: Learning Of Parities and Binary Linear Codesmentioning

confidence: 90%

“…Our method can be applied in more general setting. In particular, it could be used to prove that Jackson's original algorithm is resistant to persistent noise in MQs and was recently used to produce a noise tolerant DNF learning algorithm by Feldman et al [FGKP06].…”

Section: Boosting Weak Parity Learning Algorithm In the Presence Of Nmentioning

confidence: 99%

See 2 more Smart Citations

On Attribute Efficient and Non-adaptive Learning of Parities and DNF Expressions

Feldman

2005

Learning Theory

Self Cite

View full text Add to dashboard Cite

We consider the problems of attribute-efficient PAC learning of two well-studied concept classes: parity functions and DNF expressions over {0, 1} n . We show that attribute-efficient learning of parities with respect to the uniform distribution is equivalent to decoding highrate random linear codes from low number of errors, a long-standing open problem in coding theory.An algorithm is said to use membership queries (MQs) non-adaptively if the points at which the algorithm asks MQs do not depend on the target concept. Using a simple non-adaptive parity learning algorithm and a modification of Levin's algorithm for locating a weakly-correlated parity due to Bshouty et al., we give the first non-adaptive and attribute-efficient algorithm for learning DNF with respect to the uniform distribution. Our algorithm runs in timeÕ(ns 4 / ) and usesÕ(s 4 / ) non-adaptive MQs where s is the number of terms in the shortest DNF representation of the target concept. The algorithm improves on the best previous algorithm for learning DNF (of Bshouty et al.) and can also be easily modified to tolerate random classification noise in MQs.

show abstract

Section: Our Resultssupporting

confidence: 65%

“…FGKP06]. This allows us to relate the two learning problems directly, in particular, we obtain the following theorem.…”

Section: Learning Of Parities and Binary Linear Codesmentioning

confidence: 90%

See 1 more Smart Citation

On Attribute Efficient and Non-adaptive Learning of Parities and DNF Expressions

Feldman

2005

Learning Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…Klivans and Sherstov (2006) have recently given the first representationindependent (cryptographic) hardness results for PAC learning intersections of halfspaces. Feldman et al (2006) have obtained closely related results. The only other relevant hardness results are for representation-dependent (proper) learning: if the learner's output hypothesis must be from a restricted class of functions (e.g., intersections of halfspaces), then the learning problem in question is NP-hard with respect to randomized reductions (Alekhnovich et al 2004).…”

mentioning

confidence: 60%

Unconditional lower bounds for learning intersections of halfspaces

Klivans

Sherstov

2007

Mach Learn

View full text Add to dashboard Cite

We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statistical-query algorithm for learning the intersection of √ n halfspaces in n dimensions must make 2 Ω( √ n) queries. This is the first non-trivial lower bound on the statistical query dimension for this concept class (the previous best lower bound was n Ω(log n) ). Our lower bound holds even for intersections of low-weight halfspaces. In the latter case, it is nearly tight.We also show that the intersection of two majorities (low-weight halfspaces) cannot be computed by a polynomial threshold function (PTF) with fewer than n Ω(log n/ log log n) monomials. This is the first super-polynomial lower bound on the PTF length of this concept class, and is nearly optimal. For intersections of k = ω(log n) low-weight halfspaces, we improve our lower bound to min{2 Ω( √ n) , n Ω(k/ log k) }, which too is nearly optimal. As a consequence, intersections of even two halfspaces are not computable by polynomial-weight PTFs, the most expressive class of functions known to be efficiently learnable via Jackson's Harmonic Sieve algorithm. Finally, we report our progress on the weak learnability of intersections of halfspaces under the uniform distribution.

show abstract

“…The goal of the learner is to construct a high-accuracy hypothesis function h, i.e., one which satisfies Pr[f (x) = h(x)] ≤ ǫ where the probability is with respect to the uniform distribution and ǫ is an error parameter given to the learning algorithm. Algorithms and hardness results in this framework have interesting connections with topics such as discrete Fourier analysis [Man94], circuit complexity [LMN93], noise sensitivity and influence of variables in Boolean functions [KKL88,BKS99,KOS04,OS07], coding theory [FGKP06], privacy [BLR08,KLN + 08], and cryptography [BFKL93,Kha95]. For these reasons, and because the model is natural and elegant in its own right, the uniform distribution learning model has been intensively studied for almost two decades.…”

Section: Background and Motivationmentioning

confidence: 99%

Optimal Cryptographic Hardness of Learning Monotone Functions

Dachman-Soled

Lee

Malkin

et al. 2008

Automata, Languages and Programming

View full text Add to dashboard Cite

Abstract. A wide range of positive and negative results have been established for learning different classes of Boolean functions from uniformly distributed random examples. However, polynomial-time algorithms have thus far been obtained almost exclusively for various classes of monotone functions, while the computational hardness results obtained to date have all been for various classes of general (nonmonotone) functions. Motivated by this disparity between known positive results (for monotone functions) and negative results (for nonmonotone functions), we establish strong computational limitations on the efficient learnability of various classes of monotone functions. We give several such hardness results which are provably almost optimal since they nearly match known positive results. Some of our results show cryptographic hardness of learning polynomial-size monotone circuits to accuracy only slightly greater than 1/2 + 1/ √ n; this accuracy bound is close to optimal by known positive results (Blum et al., FOCS '98). Other results show that under a plausible cryptographic hardness assumption, a class of constant-depth, sub-polynomialsize circuits computing monotone functions is hard to learn; this result is close to optimal in terms of the circuit size parameter by known positive results as well (Servedio, Information and Computation '04). Our main tool is a complexitytheoretic approach to hardness amplification via noise sensitivity of monotone functions that was pioneered by O'Donnell (JCSS '04).

show abstract

New Results for Learning Noisy Parities and Halfspaces

Cited by 97 publications

References 40 publications

On Attribute Efficient and Non-adaptive Learning of Parities and DNF Expressions

On Attribute Efficient and Non-adaptive Learning of Parities and DNF Expressions

Unconditional lower bounds for learning intersections of halfspaces

Optimal Cryptographic Hardness of Learning Monotone Functions

Contact Info

Product

Resources

About