Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm

Abstract

Abstract. Valiant (1984) and others have studied the problem of learning various classes of Boolean functions from examples. Here we discuss incremental learning of these functions. We consider a setting in which the learner responds to each example according to a current hypothesis. Then the learner updates the hypothesis, if necessary, based on the correct classification of the example. One natural measure of the quality of learning in this setting is the number of mistakes the learner makes. For suitable classes of functions, learning algorithms are available that make a bounded number of mistakes, with the bound independent of the number of examples seen by the learner. We present one such algorithm that learns disjunctive Boolean functions, along with variants for learning other classes of Boolean functions. The basic method can be expressed as a linear-threshold algorithm. A primary advantage of this algorithm is that the number of mistakes grows only logarithmically with the number of irrelevant attributes in the examples. At the same time, the algorithm is computationally efficient in both time and space.

On Exact Learning from Random Walk

“…developed efficient algorithms for exact learning boolean threshold functions, 2-term RSE, and 2-term DNF in the RWOnline model. Those classes are already known to be learnable in the Online model [L87,FS92], but the algorithms in [BFH95] achieve a better mistake bound (for threshold functions).…”

“…developed efficient algorithms for exact learning boolean threshold functions, 2-term Ring-Sum Expansion (parity of 2 monotone terms) and 2-term DNF in the RWOnline model. Those classes are already known to be learnable in the Online model [L87,FS92] (and therefore in the RWOnline model), but the algorithm in [BFH95] for boolean threshold functions achieves a better mistake bound. They show that this class can be learned by making no more than n + 1 mistakes in the RWOnline model, improving on the O(n log n) bound for the Online model proven by Littlestone in [L87].…”

Gender Identification on Twitter Using the Modified Balanced Winnow

“…This parameter allows the Balanced Winnow to be further adjusted to fit the data optimally. The three parameters cause the Balanced Winnow algorithm to update the models and accurately determine the class of each instance [8,9]. The Balanced Winnow is based on two functions.…”

Untitled

“…The mistake-bound model of learning, introduced by Littlestone (Littlestone, 1988;1989), has attracted a considerable amount of attention (e.g., (Littleston, 1988, Littlestone & Warmuth, 1994, Blum 1994, Blum 1992a, Maass, 1991, Chen & Maass, 1994, Helmbold, Littlestone & Long, 1992, Goldman, Rivest & Schapire, 1993, Goldman & Sloan, 1994) and is recognized as one of the central models of computational learning theory. Basically it models a process of incremental learning, where the learner discovers the 'labels' of instances one by one.…”

“…We present an off-line variant of the mistake-bound model of learning. This is an intermediate model between the on-line learning model (Littlestone, 1988) and the self-directed learning model (Goldman, Rivest & Schapire, 1993, Goldman & Sloan, 1994. Just like in the other two models, a learner in the off-line model has to learn an unknown concept from a sequence of elements of the instance space on which it makes "guess and test" trials.…”