Maximum margin classifier working in a set of strings

Abstract

Numbers and numerical vectors account for a large portion of data. However, recently, the amount of string data generated has increased dramatically. Consequently, classifying string data is a common problem in many fields. The most widely used approach to this problem is to convert strings into numerical vectors using string kernels and subsequently apply a support vector machine that works in a numerical vector space. However, this nonone-to-one conversion involves a loss of information and makes it impossible to evaluate, using probability theory, the generalization error of a learning machine, considering that the given data to train and test the machine are strings generated according to probability laws. In this study, we approach this classification problem by constructing a classifier that works in a set of strings. To evaluate the generalization error of such a classifier theoretically, probability theory for strings is required. Therefore, we first extend a limit theorem for a consensus sequence of strings demonstrated by one of the authors and co-workers in a previous study. Using the obtained result, we then demonstrate that our learning machine classifies strings in an asymptotically optimal manner. Furthermore, we demonstrate the usefulness of our machine in practical data analysis by applying it to predicting proteinprotein interactions using amino acid sequences and classifying RNAs by the secondary structure using nucleotide sequences.

Optimal string clustering based on a Laplace-like mixture and EM algorithm on a set of strings

“…In this study, using the probability theory on A * developed in [19][20][21], we constructed the theory of the mixture model and the EM algorithm on A * and derived the optimal procedure for unsupervised string clustering based on the theory. We encountered the interesting phenomenon that an EM algorithm for the Laplace-like mixture on A * , which we sought, could not be written in an explicit manner because of the complex metric structure of A * , but a sequence of algorithms (i.e., a sequence of sequences of computations) that strongly consistently estimates the parameters of the Laplace-like mixture and converges to the EM algorithm with probability one was obtained explicitly.…”

“…This is different from the phenomena that an algorithm halts and that a sequence of approximate solutions from an iterative algorithm converges. The authors addressed the problem of supervised string classification by constructing a theory of a statistical learning machine that works in A * in [19]. Recently, the amount of string data has increased exponentially.…”

“…Our objects in this study are sequences of random variables that take values in A * (random strings), their distributions, and their realizations, i.e., string data. In the following sections, we use a fundamental framework of a probability theory on A * that was proposed in [20] and extended in [19,21]. A summary of this theory is provided in Subsection A1.…”

“…Similarly, statistical methods for string data should also be constructed based on probability theory on a set of strings. In this study, we address the problem of clustering string data in an unsupervised manner by applying probability theory on a set of strings developed in [19][20][21]. Our approach to unsupervised string clustering in this study is based on a mixture model of distributions on a set of strings.…”

“…Finally, in Section 5, we derive a procedure for clustering strings in an unsupervised manner from the Laplace-like mixture and describe that it is asymptotically optimal in the sense that the posterior probability of making correct classifications is maximized. A summary of probability theory on a set of strings that was developed in [19][20][21] and is used in this paper is provided in Subsection A1. In Subsection A4, we show that a result on the strong consistency of maximum likelihood estimators in the case where the parameter space is the set of real numbers (see, for example, [41,47]) can be extended to the case where the parameter space is the direct product of the sets of strings and real numbers.…”