Efficient Sequence Regression by Learning Linear Models in All-Subsequence Space

Abstract

We present a new approach for learning a sequence regression function, i.e., a mapping from sequential observations to a numeric score. Our learning algorithm employs coordinate gradient descent and Gauss-Southwell optimization in the feature space of all subsequences. We give a tight upper bound for the coordinate wise gradients of squared error loss that enables efficient Gauss-Southwell selection. The proposed bound is built by separating the positive and the negative gradients of the loss function and exploits the structure of the feature space. Extensive experiments on simulated as well as real-world sequence regression benchmarks show that the bound is effective and our proposed learning algorithm is efficient and accurate. The resulting linear regression model provides the user with a list of the most predictive features selected during the learning stage, adding to the interpretability of the method.

Anytime mining of sequential discriminative patterns in labeled sequences

“…Gsponer et al used a different approach, where a linear model is trained on a set of features extracted from sequences of the dataset [20]. Those features correspond to the search space of all possible subsequences.…”

Background Knowledge Injection for Interpretable Sequence Classification

“…In this work we adopt SEQL a linear sequence classifier algorithm, as we want to learn a model that is interpretable but still achieves high accuracy. The main idea behind SEQL is to use a greedy coordinate gradient descent with the Gauss-Southwell rule [13] which allows to avoid the explicit generation of the feature vectors [5]. A key step of this approach is the efficient search for the current best k-mer, in the sense of maximum absolute gradient value, followed by an update of the corresponding weight value β.…”

Optimization of Spaced K-mer Frequency Feature Extraction using Genetic Algorithms for Metagenome Fragment Classification

“…The main problem of using the k-mer frequency feature is dealing with a largedimension feature space when aiming to obtain high accuracy [14]. To solve this problem, Kusuma [15] introduced spaced k-mers, inspired by PatternHunter [16], to reduce the feature space dimension and improving accuracy.…”