A family of large margin linear classifiers and its application in dynamic environments

Abstract

Real-time problems, in which the learning must be fast and the importance of the features might be changing, pose a challenge to machine learning algorithms. To learn robust classifiers in such nonstationary environments, it is essential not to assign too much weight to any single feature. We solve the problems by combining regularization mechanisms with online large margin learning algorithms. We prove bounds on their error and show that removing features with small weights has little influence on the accuracy, suggesting that these methods exhibit feature selection ability. We show that such regularized learning algorithms automatically decrease the influence of the old training instances and focus on the more recent ones. This makes them especially attractive in the dynamic environments. We present experimental results on real datasets to show the merit of our algorithms.