Oracles and Queries That Are Sufficient for Exact Learning

Abstract

We show that the class of all circuits is exactly learnable in randomized expected polynomial time using weak subset and weak superset queries. This is a consequence of the following result which we consider to be of independent interest: circuits are exactly learnable in randomized expected polynomial time with equivalence queries and the aid of an NP-oracle. We also show that circuits are exactly learnable in deterministic polynomial time with equivalence queries and a P 3 -oracle. The hypothesis class for the above learning algorithms is the class of circuits of larger but polynomially related size. Also, the algorithms can be adapted to learn the class of DNF formulas with hypothesis class consisting of depth-3 7-6-7 formulas (by the work of Angluin this is optimal in the sense that the hypothesis class cannot be reduced to DNF formulas, i.e., depth-2 6-7 formulas). We also investigate the power of an NP-oracle in the context of learning with membership queries. We show that there are deterministic learning algorithms that use membership queries and an NP-oracle to learn: monotone boolean functions in time polynomial in the DNF size and CNF size of the target formula; and the class of O(log n)-DNF & O(log n)-CNF formulas in time polynomial in n. We also show that, with an NP-oracle and membership queries, there is a randomized expected polynomial time algorithm that learns any class that is learnable from membership queries with unlimited computational power. Using similar techniques, we show the following both for membership and for equivalence queries (when the hypotheses allowed are precisely the concepts in the class); any class learnable with unbounded computational-power is learnable in deterministic polynomial time with a p 5 -oracle. Furthermore, we identify the combinatorial properties that completely determine learnability in this information-theoretic sense. Finally we point out a consequence of our result in structural complexity theory showing that if every NP set has polynomial-size circuits then the polynomial hierarchy collapses to ZPPNP . ] 1996Academic Press, Inc.

Quantum versus classical learnability

“…Our results draw on lower bound techniques from both quantum computation and computational learning theory [2,5,6,8,12,24]. A detailed description of the relationship between our results and previous work on quantum versus classical black-box query complexity is given in Section 3.4.…”

Parameterized Learnability of k-Juntas and Related Problems

“…Our main result here is that any such class is randomized fpt EQ-learnable with access to an oracle in W[P], provided that the Hamming weight is used as parameter. Our learning algorithm uses a similar strategy as the exact learning algorithm of Bshouty et al [7]. We first recall a version of the Valiant-Vazirani lemma [24] that lower bounds the probability that a randomly chosen linear function h isolates some x ∈ D (we say that a function h : {0, 1} s → {0, 1} l isolates x in D ⊆ {0, 1} s , if x is the only string in D with h(x) = 0 l ).…”

“…Actually, we prove a more general result: we consider the problem of learning parameterized concept classes for which the membership of an assignment to a given concept is decidable in FPT and show that these concept classes are exactly learnable by a randomized fpt algorithm with equivalence queries and with access to a W[P] oracle, provided that the Hamming weight is used as parameter. Our learning algorithm uses a similar strategy as the algorithm designed by Bshouty et al [7] for exactly learning boolean circuits with equivalence queries and with the help of an NP oracle.…”

Untitled

“…Furthermore, it is known that any algorithm that learns the class of monotone functions with membership queries must pose (max{|cnf( f )|, |dnf( f )|}) queries (where f is the monotone function to be compiled) (Korobkov, 1965;Bshouty et al, 1996). The result also applies to the class of monotone functions representable as read-k CNF formulas when k ≥ 2.…”