Generalized Similarity Kernels for Efficient Sequence Classification

Abstract

Kernel-based approaches for sequence classification have been successfully applied to a variety of domains, including the text categorization, image classification, speech analysis, biological sequence analysis, time series and music classification, where they show some of the most accurate results.Typical kernel functions for sequences in these domains (e.g., bag-of-words, mismatch, or subsequence kernels) are restricted to discrete univariate (i.e. one-dimensional) string data, such as sequences of words in the text analysis, codeword sequences in the image analysis, or nucleotide or amino acid sequences in the DNA and protein sequence analysis. However, original sequence data are often of real-valued multivariate nature, i.e. are not univariate and discrete as required by typical k-mer based sequence kernel functions.In this work, we consider the problem of the multivariate sequence classification (e.g., classification of multivariate music sequences, or multidimensional protein sequence representations). To this end, we extend univariate kernel functions typically used in sequence domains and propose efficient multivariate similarity kernel method (MVDFQ-SK) based on (1) a direct feature quantization (DFQ) of each sequence dimension in the original real-valued multivariate sequences and (2) applying novel multivariate discrete kernel measures on these multivariate discrete DFQ sequence representations to more accurately capture similarity relationships among sequences and improve classification performance.Experiments using the proposed MVDFQ-SK kernel method show excellent classification performance on three challenging music classification tasks as well as protein sequence classification with significant 25-40% improvements over univariate kernel methods and existing state-of-the-art sequence classification methods.

Maximum margin classifier working in a set of strings

“…Many previous studies proved the kernel-based methods for string representations to be useful in practical string data analysis. See, for example, [26,[43][44][45][46][47][48][49][50][51][52] for important recent studies on the kernel-based methods for string data. However, it is impossible to evaluate, applying traditional probability theory that has been constructed on spaces such as the Euclidean space R p and Hilbert space L 2 , the performance of the methods based on the string kernels, considering that a sample of observed strings is a part of a population generated according to a probability law.…”

Timbral modeling for music artist recognition using i-vectors

“…The third baseline (BLsignature) generates compact signatures for each music track using a 15-dimensional MFCC feature set and compares these using bipartite graph matching [10]. The fourth baseline (BLmultivar) uses multivariate kernels [11] with the direct uniform quantization of the 13-dimensional MFCC features. The results for the latter three are taken from their publications, while the results for BLGMM baseline are reproduced using the implementation provided with the dataset.…”

“…For the 13-dimensional MFCCs, we use MIRTOOLBOX [29] with 40 frequency bands, 25 ms window length and 50% overlap to extract features from 32kbps mp3 files provided in the dataset. This is because another baseline method [11] also used it to extract 13-MFCCs. We use 2000 randomly-selected frames from the middle area of each song to compute Baum-Welch statistics, assuming that this middle area of the song contains the most singing voice data.…”

“…Compact signatures are generated for a song in [10], then are compared using bipartite graph matching. Also, multivariate kernels [11] have been used to build a model of an artist and assign songs to artists using a sequence of features, with promising results.…”

Estimating Descriptors for Large Graphs

“…Typically, the kernel value is the inner-product between two feature vectors corresponding to the two graphs. This so-called kernel trick has been used successfully to evaluate pairwise of other structures such as images and sequences [4,10,12]. Several graph kernels based on sub-structural patterns have been proposed, such as the Shortest-Path [5] and Graphlet [17] kernels.…”