A New Method for Symbolic Sequences Analysis. An Application to Long Sequences

Abstract

Abstract:The method for symbolic sequence decomposition into a set of consecutive, distinct, non-overlapping strings of various lengths is proposed. Representation of the sequence as a set of words allows one to use set theory notions. The main result is a quite new definition of the similarity between any two sequences over a given alphabet. No prior sequence alignment is necessary. In the present paper two applications of a set of words are described. In the first a similarity measure is applied to prepare centroids for K-means algorithm. It results in a high performance grouping method for long DNA sequences. The other application concerns the statistical analysis of word attributes. It is shown that similarity, complexity and correlation function of word attributes across sequences of digits of fractional parts of some irrational numbers support the suggestion that the sequences are instances of a random sequence of decimal digits.

Numerical Representation of Symbolic Data

“…has been introduced in [7]. The value of similarity varies between 0 when the spectra are disjoint sets and 1 when sequence C 1 and C 2 are mutual copies.…”

“…In Ref. [7] the sets of most similar pairs, fours or eights sequences as the initialising set were discussed. The average vector of each set was considered as the starting centroid location.…”

Comment on the paper “A New Method for Symbolic Sequences Analysis. An Application to Long Sequences”

“…In the recent issue of CMST there appeared an interesting paper [1] by B. Kozarzewski on the new method for symbolic sequences analysis. This method was tested on several long sequences, in particular on the digits of the so called "Champernowne number", which we will denote below as C 10 .…”

“….. E. Borel had proved in 1909 [3] that almost all real numbers are normal and the first explicite example of normal number was C 10 . On page 98 of [1], right column, it is written about C 10 "The number is assumed to be transcendental". In fact, the number is transcendental as it follows from the more general theorem proved by K. Mahler in [4].…”

“…The plot showing share of digits 1, 4 and 8 among the first 2000000 digits of C10. There is a half million plotted points: digit frequency every 4-th position in C10 is plotted duce the original plot from[1] this time with every 4-th point plotted, i.e. for each digit 1, 4, 8 there are 500000 points on the graphs.…”