Inhomogeneous Parsimonious Markov Models

Abstract

Abstract. We introduce inhomogeneous parsimonious Markov models for modeling statistical patterns in discrete sequences. These models are based on parsimonious context trees, which are a generalization of context trees, and thus generalize variable order Markov models. We follow a Bayesian approach, consisting of structure and parameter learning. Structure learning is a challenging problem due to an overexponential number of possible tree structures, so we describe an exact and efficient dynamic programming algorithm for finding the optimal tree structures.We apply model and learning algorithm to the problem of modeling binding sites of the human transcription factor C/EBP, and find an increased prediction performance compared to fixed order and variable order Markov models. We investigate the reason for this improvement and find several instances of context-specific dependences that can be captured by parsimonious context trees but not by traditional context trees.

Inferring intra-motif dependencies of DNA binding sites from ChIP-seq data

“…Even though not all TFs can be expected to bind to such a long binding site, adding some possibly uninformative positions is less harmful than not being able to take into account all informative positions. As a motif model we use inhomogeneous parsimonious Markov models [ 34 ] of order 0–4, which also includes the standard PWM model [ 1 , 2 ] that is equivalent to the PMM of order zero.…”

“…For estimating a sequence motif from limited training data, e.g., a few experimentally verified binding sites, the PWM model will certainly remain the optimal choice. For fully observable data, more than 10 2 training sequences are typically required before any model with more parameters than a PWM model can effectively utilize additional information in the data [ 34 , 42 ]. Moreover, due to its small parameter space, the PWM model is more robust to low-quality data, where models that attempt to infer dependencies may be more prone to adapting to noise.…”

“…As motif model, we use an inhomogeneous parsimonious Markov model (PMM) [ 34 ], which is a model that makes a position specific use of parsimonious context trees [ 35 ] (PCTs) for taking into account statistical dependencies while keeping the parameter space small. A PCT of depth d is a rooted, balanced tree that represents all context sequences of length d and organizes them in different groups, represented by the leaves in the tree.…”

“…In such a case, when all binding sites are already known and pre-aligned, i.e., experimentally confirmed binding sites or data sets from a database such as JASPAR [ 8 ], the task of learning PMMs involves no latent variables at all and can thus be carried out exactly. While the first approach for learning PMMs from fully observable data was still based on a Bayesian approach with massive hyperparameter-tuning [ 34 ], it could be later shown that this effort might not be necessary [ 42 ]. The combination of the Bayesian information criterion (BIC) [ 46 ] as structure score for finding the optimal PCT structures and the factorized sequential Normalized Maximum Likelihood (fsNML) [ 47 ] for estimating the conditional probability parameters appears to constitute a robust alternative [ 42 ].…”

“…Recently, inhomogeneous parsimonious Markov models (PMMs) [ 34 ] have been proposed with the aim of allowing a fine-grained position-specific adaption of model complexity based on observed data. PMMs contain as special case the simple PWM model [ 1 , 2 ], the weight array model [ 3 ], traditional inhomogeneous Markov models of higher order, and variable order Markov models [ 6 ], but their structural flexibility reaches further.…”