A learning algorithm for the longest common subsequence problem

Abstract

We present an experimental study of a learning algorithm for the longest common subsequence problem, LCS . Given an arbitrary input domain, the algorithm learns an LCS -procedure tailored to that domain. The learning is done with the help of an oracle, which can be any LCS -algorithm. After solving a limited number of training inputs using an oracle, the learning algorithm outputs a new LCS -procedure.Our experiments demonstrate that, by allowing a slight loss of optimality, learning yields a procedure which is significantly faster than the oracle. The oracle used for the experiments is the np -procedure by Wu et al. , a modification of Myers' classical LCS -algorithm. We show how to scale up the results of learning on small inputs to inputs of arbitrary lengths. For the domain of two random 2-symbol inputs of length n , learning yields a program with 0.999 expected accuracy, which runs in O ( n 1.41 )-time, in contrast with O ( n 2 /log n ) running time of the fastest theoretical algorithm that produces optimal solutions. For the domain of random 2-symbol inputs of length 100,000, the program runs 10.5 times faster than the np -procedure, producing 0.999- accurate outputs. The scaled version of the evolved algorithm applied to random inputs of length 1 million runs approximately 30 times faster than the np -procedure while constructing 0.999- accurate solutions. We apply the evolved algorithm to DNA sequences of various lengths by training on random 4-symbol sequences of up to length 10,000. The evolved algorithm, scaled up to the lengths of up to 1.8 million, produces solutions with the 0.998-accuracy in a fraction of the time used by the np .

Discovering optimization algorithms through automated learning

“…Database Size [28,29,42,47,50]. A learning approach for designing faster LCS-algorithms was used in [13]. Similar to learning for backtracking, learning for dynamic programming attempts to discover the "search area" which is essential for finding solutions to instances from the input domain in question.…”

“…Learning, as implemented in [13], consists of three major steps: (a) generating uniformly at random a finite number of instances from a given domain; (b) applying an oracle to each instance, in order to produce a set of solution traces; and (c) refining and scaling the search area. The algorithm which emerges from learning implements the restricted dynamic programming strategy, for which the full search is reduced to the search within the search area learned.…”

“…By this approach, we learn domain-specific algorithms by consulting an oracle, which is also an optimization algorithm for the problem in question. Designing optimization algorithms by means of supervised learning was initiated in [4,24,19] and was further applied in [11,12,13,20,21,22,23]. Briefly, the PAC model applied to learning combinatorial algorithms is comprised of four stages:…”

“…For example, in order to design an algorithm for the longest common subsequence problem [13], we use the algorithm from [50] as an oracle. To design a fast heuristic for the Maximum Clique Problem [19], we use a procedure which implements a restricted backtracking algorithmic paradigm.…”

“…Another way of avoiding the time-consuming learning stage of the design process is to do the learning on inputs of a small to moderate size, and "generalize" (scale up) the results to the inputs of a large size. This technique is successfully used in [13].…”