Testing Halfspaces

Abstract

This paper addresses the problem of testing whether a Boolean-valued function f is a halfspace, i.e. a function of the form f (x) = sgn(w · x − θ). We consider halfspaces over the continuous domain R n (endowed with the standard multivariate Gaussian distribution) as well as halfspaces over the Boolean cube {−1, 1} n (endowed with the uniform distribution). In both cases we give an algorithm that distinguishes halfspaces from functions that are ǫ-far from any halfspace using only poly( 1 ǫ ) queries, independent of the dimension n.Two simple structural results about halfspaces are at the heart of our approach for the Gaussian distribution: the first gives an exact relationship between the expected value of a halfspace f and the sum of the squares of f 's degree-1 Hermite coefficients, and the second shows that any function that approximately satisfies this relationship is close to a halfspace. We prove analogous results for the Boolean cube {−1, 1} n (with Fourier coefficients in place of Hermite coefficients) for balanced halfspaces in which all degree-1 Fourier coefficients are small. Dealing with general halfspaces over {−1, 1} n poses significant additional complications and requires other ingredients. These include "cross-consistency" versions of the results mentioned above for pairs of halfspaces with the same weights but different thresholds; new structural results relating the largest degree-1 Fourier coefficient and the largest weight in unbalanced halfspaces; and algorithmic techniques from recent work on testing juntas [FKR + 02].

A Robust Khintchine Inequality, and Algorithms for Computing Optimal Constants in Fourier Analysis and High-Dimensional Geometry

“…Our second proof of Theorem 1 employs several sophisticated ingredients from recent work on structural properties of LTFs [OS11,MORS10]. The first of these ingredients is a result (Theorem 6.1 of [OS11]) which essentially says that any LTF f (x) = sign(w · x) can be perturbed very slightly to another LTF f (x) = sign(w · x) (where both w and w are unit vectors).…”

“…It turns out that the anti-concentration of w · x, together with results on the degree-1 Fourier spectrum of "regular" halfspaces from [MORS10], lets us establish a lower bound on W ≤1 [f ] that is strictly greater than 1/2. Then the fact that f and f agree on almost every input in {−1, 1} n lets us argue that the original LTF f must similarly have W ≤1 [f ] strictly greater than 1/2.…”

Approximating the Influence of Monotone Boolean Functions in $O(\sqrt{n})$ Query Complexity

“…The influence of variables plays an implicit role in many learning algorithms, and in particular those that build on Fourier analysis, beginning with [25]. 3 If one wants an additive error of ǫ, then Ω((n/ǫ) 2 ) queries are necessary (when the influence is large) [27]. 4 We also note that in the case of monotone functions, the total influence equals twice the sum of the Fourier coefficients…”

“…The influence of functions has played a central role in several areas of computer science. In particular, this is true for distributed computing (e.g., [2,21]), hardness of approximation (e.g., [12,22]), learning theory (e.g., [18,9,29,30,10]) 2 and property testing (e.g., [13,4,5,26,31]). The notion of influence also arises naturally in the context of probability theory (e.g., [32,33,3]), game theory (e.g., [24]), reliability theory (e.g., [23]), as well as theoretical economics and political science (e.g., [1,19,20]).…”

Distribution-Free Testing Algorithms for Monomials with a Sublinear Number of Queries

“…There is quite a large variety of function classes for which the complexity of testing is strictly lower than that of learning when the underlying distribution is uniform (e.g., linear functions [BLR93], low-degree polynomials [RS96], singletons, monomials [PRS02] and small monotone DNF [PRS02], monotone functions (e.g., [EKK + 00, DGL + 99]), small juntas [FKR + 04], small decision lists, decision trees and (general) DNF [DLM + 07] linear threshold functions [MORS09], and more). In contrast, there are relatively few such positive results for distribution-free testing [HK03,HK04,HK07], and, in general, designing distributionfree testing algorithms tends to be more challenging.…”