Vacillatory and BC learning on noisy data

Case, John; Jain, Sanjay; Stephan, Frank

doi:10.1007/3-540-61863-5_53

Cited by 7 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding the names of the learning criteria studied in the present paper, originally [31] "Ex" stood for "explanatory," "Fex" stood for "finitely explanatory," and Bc for "behaviorally correct." tion, however, that there are some interesting effects on learning power for vacillatory function learning wrought by bounding suitably sensitive measures of the computational complexity of the learning functions themselves [24] and by the introduction of noisy input data [25].…”

mentioning

confidence: 99%

The Power of Vacillation in Language Learning

Case¹

1999

SIAM J. Comput.

Self Cite

View full text Add to dashboard Cite

Some extensions are considered of Gold's influential model of language learning by machine from positive data. Studied are criteria of successful learning featuring convergence in the limit to vacillation between several alternative correct grammars. The main theorem of this paper is that there are classes of languages that can be learned if convergence in the limit to up to (n + 1) exactly correct grammars is allowed but which cannot be learned if convergence in the limit is to no more than n grammars, where the no more than n grammars can each make finitely many mistakes. This contrasts sharply with results of Barzdin and Podnieks and, later, Case and Smith for learnability from both positive and negative data. A subset principle from a 1980 paper of Angluin is extended to the vacillatory and other criteria of this paper. This principle provides a necessary condition for avoiding overgeneralization in learning from positive data. It is applied to prove another theorem to the effect that one can optimally eliminate half of the mistakes from final programs for vacillatory criteria if one is willing to converge in the limit to infinitely many different programs instead. Child language learning may be sensitive to the order or timing of data presentation. It is shown, though, that for the vacillatory success criteria of this paper, there is no loss of learning power for machines which are insensitive to order in several ways simultaneously. For example, partly set-driven machines attend only to the set and length of sequence of positive data, not the actual sequence itself. A machine M is weakly n-ary order independent def ⇔ for each language L on which, for some ordering of the positive data about L, M converges in the limit to a finite set of grammars, there is a finite set of grammars D (of cardinality ≤ n) such that M converges to a subset of this same D for each ordering of the positive data for L. The theorem most difficult to prove in the paper implies that machines which are simultaneously partly set-driven and weakly n-ary order independent do not lose learning power for converging in the limit to up to n grammars. Several variants of this theorem are obtained by modifying its proof, and some of these variants have application in this and other papers. Along the way it is also shown, for the vacillatory criteria, that learning power is not increased if one restricts the sequence of positive data presentation to be computable. Some of these results are nontrivial lifts of prior work for the n = 1 case done by the Blums; Wiehagen; Osherson, Stob, and Weinstein; Schäfer; and Fulk.

show abstract

mentioning

confidence: 99%

The Power of Vacillation in Language Learning

Case¹

1999

SIAM J. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Case, Jain and Sharma (1997) present positive and negative results regarding synthesizers of language learners which tolerate noisy data, where noise is modeled as in Stephan (1995), Case, Jain and Stephan (1996). Furthermore, the proofs of the positive results provide characterizations of corresponding noisetolerantly learnable language classes.…”

mentioning

confidence: 99%

The Synthesis of Language Learners

Baliga

Case

Jain

1999

Information and Computation

Self Cite

View full text Add to dashboard Cite

An index for an r.e. class of languages (by definition) is a procedure which generates a sequence of grammars defining the class. An index for an indexed family of languages (by definition) is a procedure which generates a sequence of decision procedures defining the family.Studied is the metaproblem of synthesizing from indices for r.e. classes and for indexed families of languages various kinds of language-learners for the corresponding classes or families indexed.Many positive results, as well as some negative results, are presented regarding the existence of such synthesizers. The negative results essentially provide lower bounds for the positive results. The proofs of some of the positive results yield, as pleasant corollaries, subset-principle or tell-tale style characterizations for the learnability of the corresponding classes or families indexed.For example, the indexed families of recursive languages that can be behaviorally correctly identified from positive data are surprisingly characterized by Angluin's (1980b) Condition 2 (the subset principle for circumventing overgeneralization). 2 The problem is not that correct grammars for finite classes of languages can't be learned in the limit; they can (Osherson, Stob and Weinstein (1986a)), and by an obvious enumeration technique. The problem is how to pass algorithmically from a list of grammars to a machine which so learns the corresponding languages.A study of the proof of the result shows that, intuitively, the difficulty, given a pair of grammars g 1 , g 2 for a language class L = {L 1 , L 2 }, to synthesize a TxtEx-learner successful on L is in deciding from g 1 , g 2 whether or not L 1 = L 2 . This equivalence problem is well-known to be algorithmically unsolvable (Rogers (1967)).3 Bc is short for behaviorally correct. 4

show abstract

“…Since, from a machine M, one can effectively construct a machine M which NoisyInfBc a -identifies NoisyInfEx 2a (M) (see [CJS96]), we immediately have (using Corollary 4 for the * -case) the following result about effective synthesis for NoisyInfBc a -identification.…”

Section: When Effective Synthesis Is Possiblementioning

confidence: 95%

“…As we will see, in the present paper, the proofs of most of our positive results which provide the existence of learner-synthesizers which synthesize noise-tolerant learners also yield pleasant characterizations which look like strict versions of the subset principle (1). 4 We consider language learning from both texts (only positive data) and from informants (both positive and negative data), and we adopt Stephan's [Ste95,CJS96] noise model for the present study. Roughly, in this model correct information about an object occurs infinitely often while incorrect information occurs only finitely often.…”

Section: Introductionmentioning

confidence: 99%

Synthesizing noise-tolerant language learners

Case

Jain

Sharma

2001

Theoretical Computer Science

Self Cite

View full text Add to dashboard Cite

An index for an r.e. class of languages (by definition) generates a sequence of grammars defining the class. An index for an indexed family of languages (by definition) generates a sequence of decision procedures defining the family. F. Stephan's model of noisy data is employed, in which, roughly, correct data crops up infinitely often, and incorrect data only finitely often. Studied, then, is the synthesis from indices for r.e. classes and for indexed families of languages of various kinds of noise-tolerant language-learners for the corresponding classes or families indexed. Many positive results, as well as some negative results, are presented regarding the existence of such synthesizers. The proofs of most of the positive results yield, as pleasant corollaries, strict subset-principle or telltale style characterizations for the noise-tolerant learnability of the corresponding classes or families indexed.

show abstract

Vacillatory and BC learning on noisy data

Cited by 7 publications

References 26 publications

The Power of Vacillation in Language Learning

The Power of Vacillation in Language Learning

The Synthesis of Language Learners

Synthesizing noise-tolerant language learners

Contact Info

Product

Resources

About