Grammatical inference as a principal component analysis problem

Bailly, Raphaël; Denis, François; Ralaivola, Liva

doi:10.1145/1553374.1553379

Cited by 35 publications

(46 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In substantial breakthroughs, Mossel and Roch [33], Denis et al [22], Hsu et al [29], and Bailly et al [4] gave algorithms having formal proofs of PAC learning the full class of pfa. The sample size and running times of the algorithms depend polynomially in the inverse of some quantity of a spectral flavor associated to the Hankel matrix of the target distribution.…”

Section: A Panoramic View Of Known Resultsmentioning

confidence: 99%

“…Then, we specialize the algorithm to pfa and the case in which the input is a randomly drawn finite sample. We discuss the solutions given by Denis et al [22] on the one hand and, on the other, by Mossel and Roch [33], Hsu et al [29], and Bailly et al [4]. The latter leads to the spectral method, which we expose in more detail following mostly the presentation in [9].…”

Section: Learning Pfamentioning

confidence: 99%

“…The solution taken by e.g., [33,29,4], which leads to the spectral method, is less combinatorial and more algebraic, less local and more global, and can be phrased as follows: Let us instead find a matrix H that 1) is easy to compute 2) has the same dimensions as H, but rank at most n, and 3) is "as close as possible" toĤ under some metric, with this rank constraint. One particular way of computing such H (among, perhaps, other possibilities) is the spectral method described next.…”

Section: An Oracle Algorithm For Learning Wamentioning

confidence: 99%

See 2 more Smart Citations

Learning Probability Distributions Generated by Finite-State Machines

Castro

Gavaldà

2016

Topics in Grammatical Inference

View full text Add to dashboard Cite

We review methods for inference of probability distributions generated by probabilistic automata and related models for sequence generation. We focus on methods that can be proved to learn in the inference in the limit and PAC formal models. The methods we review are state merging and state splitting methods for probabilistic deterministic automata and the recently developed spectral method for nondeterministic probabilistic automata. In both cases, we derive them from a high-level algorithm described in terms of the Hankel matrix of the distribution to be learned, given as an oracle, and then describing how to adapt that algorithm to account for the error introduced by a finite sample.

show abstract

Section: A Panoramic View Of Known Resultsmentioning

confidence: 99%

Section: Learning Pfamentioning

confidence: 99%

Section: An Oracle Algorithm For Learning Wamentioning

confidence: 99%

See 1 more Smart Citation

Learning Probability Distributions Generated by Finite-State Machines

Castro

Gavaldà

2016

Topics in Grammatical Inference

View full text Add to dashboard Cite

show abstract

“…-Other algorithms learning multiplicity automata have been developed, using common approaches in machine learning such as recurrent neural networks [12], Principal Component Analysis [4] or a spectral approach [3].…”

Section: How To Learn a Probabilistic Automaton?mentioning

confidence: 99%

PAutomaC: a probabilistic automata and hidden Markov models learning competition

2013

View full text Add to dashboard Cite

Approximating distributions over strings is a hard learning problem. Typical techniques involve using finite state machines as models and attempting to learn these; these machines can either be hand built and then have their weights estimated, or built by grammatical inference techniques: the structure and the weights are then learned simultaneously. The PAutomaC competition, run in 2012, was the first challenge to allow comparison between methods and algorithms and built a first state of the art for these techniques. Both artificial data and real data were proposed and contestants were to try to estimate the probabilities of new strings. The purpose of this paper is to describe some of the technical and intrinsic challenges such a competition has to face, to give a broad state of the art concerning both challenges dealing with learning grammars and finite state machines and the relevant literature. This paper also provides the results of the competition and a brief description and analysis of the different approaches the main participants used.

show abstract

“…Other approaches to language learning, also based on linear algebra, can be found in the literature, e.g. [9,2].…”

Section: Introductionmentioning

confidence: 99%

A Spectral Learning Algorithm for Finite State Transducers

Balle

Quattoni

Carreras

2011

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Finite-State Transducers (FSTs) are a popular tool for modeling paired input-output sequences, and have numerous applications in real-world problems. Most training algorithms for learning FSTs rely on gradient-based or EM optimizations which can be computationally expensive and suffer from local optima issues. Recently, Hsu et al. [13] proposed a spectral method for learning Hidden Markov Models (HMMs) which is based on an Observable Operator Model (OOM) view of HMMs. Following this line of work we present a spectral algorithm to learn FSTs with strong PAC-style guarantees. To the best of our knowledge, ours is the first result of this type for FST learning. At its core, the algorithm is simple, and scalable to large data sets. We present experiments that validate the effectiveness of the algorithm on synthetic and real data.

show abstract

Grammatical inference as a principal component analysis problem

Cited by 35 publications

References 4 publications

Learning Probability Distributions Generated by Finite-State Machines

Learning Probability Distributions Generated by Finite-State Machines

PAutomaC: a probabilistic automata and hidden Markov models learning competition

A Spectral Learning Algorithm for Finite State Transducers

Contact Info

Product

Resources

About