Algorithms for learning parsimonious context trees

Abstract

Parsimonious context trees, PCTs, provide a sparse parameterization of conditional probability distributions. They are particularly powerful for modeling context-specific independencies in sequential discrete data. Learning PCTs from data is computationally hard due to the combinatorial explosion of the space of model structures as the number of predictor variables grows. Under the score-and-search paradigm, the fastest algorithm for finding an optimal PCT, prior to the present work, is based on dynamic programming. While the algorithm can handle small instances fast, it becomes infeasible already when there are half a dozen fourstate predictor variables. Here, we show that common scoring functions enable the use of new algorithmic ideas, which can significantly expedite the dynamic programming algorithm on typical data. Specifically, we introduce a memoization technique, which exploits regularities within the predictor variables by equating different contexts associated with the same data subset, and a bound-and-prune technique, which exploits regularities within the response variable by pruning parts of the search space based on score upper bounds. On real-world data from recent applications of PCTs within computational biology the ideas are shown to reduce the traversed search space and the computation time by several orders of magnitude in typical cases.

Disentangling transcription factor binding site complexity

“…This model is the core of the InMoDe tools ( 24 ), examples are shown in Figures 1C and 2C . For this model, learning additionally requires selecting PCTs at each position based on the data for which we employ a sophisticated dynamic programming algorithm ( 35 ).…”

Computation of exact probabilities associated with overlapping pattern occurrences

“…VLMC have also been called Markov sources (Rissanen, ; Roos & Yu, ), finite‐memory sources (Weinberger, Lempel, & Ziv, ; Weinberger, Rissanen, & Feder, ), variable‐order Markov models (Begleiter, El‐Yaniv, & Yona, ), probabilistic suffix trees (Gabadinho & Ritschard, ), and probabilistic suffix automata (Ron, Singer, & Tishby, ) in the literature. A generalization of context trees is parsimonious context trees (PCT) (Bourguignon & Robelin, ; see also Eggeling, Koivisto, & Grosse, and Eggeling & Koivisto, ). PCT allow nodes of the tree to be a subset of Σ.…”

InMoDe: tools for learning and visualizing intra-motif dependencies of DNA binding sites

“…Both model structure and parameters of an iPMM can be robustly learned without resorting to computationally expensive parameter tuning even when latent variables are involved ( Eggeling et al , 2015 ). While finding optimal PCTs is computationally hard, recent algorithmic advances allow us to solve typical instances fast enough to effectively consider dependencies up to order six ( Eggeling and Koivisto, 2016 ). InMoDe also provides tools for applying learned models, such as scanning given sequences for statistically significant hits and classifying binding sites.…”