Learning nonsingular phylogenies and hidden Markov models

Abstract

In this paper we study the problem of learning phylogenies and hidden Markov models. We call a Markov model nonsingular if all transition matrices have determinants bounded away from 0 (and 1). We highlight the role of the nonsingularity condition for the learning problem. Learning hidden Markov models without the nonsingularity condition is at least as hard as learning parity with noise, a wellknown learning problem conjectured to be computationally hard. On the other hand, we give a polynomial-time algorithm for learning nonsingular phylogenies and hidden Markov models.1. Introduction. In this paper we consider the problem of learning phylogenies and hidden Markov models, two of the most popular Markov models used in applications.Phylogenies are used in evolutionary biology to model the stochastic evolution of genetic data on the ancestral tree relating a group of species. More precisely, the leaves of the tree correspond to (known) extant species. Internal nodes represent extinct species, while the root of the tree represents the most recent ancestor to all species in the tree. Following paths from the root to the leaves, each bifurcation indicates a speciation event whereby two new species are created from a parent.The underlying assumption is that genetic information evolves from the root to the leaves according to a Markov model on the tree. This genetic information may consist of DNA sequences, proteins, and so on. Suppose,

Spectral learning of weighted automata

“…A more ambitious problem is that of grammatical inference, where the goal is to induce the model only from sequences. Regarding spectral methods, Mossel and Roch (2005) study the induction of the topology of a phylogenetic tree-shaped model, and Hsu et al (2012) discuss spectral techniques to induce PCFG, with dependency grammars as a special case.…”

Robust Estimators in High Dimensions without the Computational Intractability

“…A central goal of machine learning is to design efficient algorithms for fitting a model to a collection of observations. In recent years, there has been considerable progress on a variety of problems in this domain, including algorithms with provable guarantees for learning mixture models [1], [2], [3], [4], [5], phylogenetic trees [6], [7], HMMs [8], topic models [9], [10], and independent component analysis [11]. These algorithms crucially rely on the assumption that the observations were actually generated by a model in the family.…”

Learning mixtures of product distributions over discrete domains

“…Both [14] and [8] stated as an open question the problem of obtaining a polynomial-time algorithm for learning a mixture of k > 2 product distributions. Indeed, recent work of Mossel and Roch [20] on learning phylogenetic trees argues that the rank-deficiency of transition matrices is a major source of difficulty, and this may indicate why k = 2 has historically been a barrier-a two-row matrix can be rank-deficient only if one row is a multiple of the other, whereas the general case of k > 2 is much more complex.…”

“…work centers on trying to find the exact best mixture model (in terms of likelihood) which explains a given data sample; this is computationally intractable in general. In contrast, our main goal (and the goal of [18,14,9,8,20]) is to obtain efficient algorithms that produce -close hypotheses.…”

“…Following the same logic for Q * where KL(P||Q * ) ≤ , we get Pr[Λ(Q * ) ≥ m(H(P) + 2 )] = Pr[Λ(Q * ) ≥ m(H(P) + ) + m ] ≤ Pr[Λ(Q * ) ≥ m(H(P) + KL(P||Q * )) + m ] = Pr[Λ(Q * ) ≥ E S [Λ(Q * )] + m ] (β/α) . (21) Theorem 16 follows from plugging(20) and(21) into(17).…”