Budget Online Learning Algorithm for Least Squares SVM

Jian, Ling; Shen, Shu-Qian; Li, Jundong; Liang, Xijun; Li, Lei

doi:10.1109/tnnls.2016.2574332

Cited by 22 publications

(14 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the number of new samples, which poses both computational and memory issues especially for data streaming. Motivated by the online budgeted learning algorithms [15]- [20], we propose a budgeted online KRR (abbr. BOKRR) in which the number of input vectors in the active set is bounded by a predefined number, i.e., budget size B.…”

Section: B Budgeted Online Kernel Ridge Regressionmentioning

confidence: 99%

“…The algorithm applying this strategy would be named as Sliding Windows Online KRR hereinafter. Reference [15] presents a strategy based on maximum similarity criterion. It removes the sample that is most similar to the current active set.…”

Section: Budget Maintenancementioning

confidence: 99%

“…Researchers have also presented numerous techniques to handle the problems occurred in the scenario of online learning, especially the computational and memory issues. An effective technique to address the computational and memory issues is reducing the number of required vectors [15], [16], [22], [34]- [36]. Arce and Salinas [22] propose an online RR method based on sliding windows technology.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Toward Budgeted Online Kernel Ridge Regression on Streaming Data

et al. 2019

Self Cite

View full text Add to dashboard Cite

''Concept drift'' makes learning from streaming data fundamentally different from traditional batch learning. Focusing on the regression task on streaming data, this paper presents an efficient online learning algorithm, i.e., budgeted online kernel ridge regression (BOKRR). It is a budget version kernel ridge regression algorithm coupled with minimum contribution criterion to maintain the budget of the active set. BOKRR employs low-rank correction technology and the Sherman-Morrison-Woodbury formula to update the dynamic KRR model with the computational complexity of only O(B 2) with B learning samples (Budget size of an active set). Limited storage burden and efficient computational ability make the proposed BOKRR algorithm an ideal candidate to process streaming data. The experimental results on benchmark and real-world datasets further demonstrate the validity and efficiency of the proposed algorithms. INDEX TERMS Budget, kernel ridge regression, online learning, streaming data.

show abstract

Section: B Budgeted Online Kernel Ridge Regressionmentioning

confidence: 99%

Section: Budget Maintenancementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Toward Budgeted Online Kernel Ridge Regression on Streaming Data

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Instead of training the classifier again from scratch on the combined training set, the online learning algorithm incrementally updates the classifier to incorporate new examples into the learning model [21]. In this way, online learning can significantly save on computation costs and be more suitable to deal with large scale problems.…”

Section: Related Workmentioning

confidence: 99%

A Noise-Resilient Online Learning Algorithm for Scene Classification

et al. 2018

Self Cite

View full text Add to dashboard Cite

The proliferation of remote sensing imagery motivates a surge of research interest in image processing such as feature extraction and scene recognition, etc. Among them, scene recognition (classification) is a typical learning task that focuses on exploiting annotated images to infer the category of an unlabeled image. Existing scene classification algorithms predominantly focus on static data and are designed to learn discriminant information from clean data. They, however, suffer from two major shortcomings, i.e., the noisy label may negatively affect the learning procedure and learning from scratch may lead to a huge computational burden. Thus, they are not able to handle large-scale remote sensing images, in terms of both recognition accuracy and computational cost. To address this problem, in the paper, we propose a noise-resilient online classification algorithm, which is scalable and robust to noisy labels. Specifically, ramp loss is employed as loss function to alleviate the negative affect of noisy labels, and we iteratively optimize the decision function in Reproducing Kernel Hilbert Space under the framework of Online Gradient Descent (OGD). Experiments on both synthetic and real-world data sets demonstrate that the proposed noise-resilient online classification algorithm is more robust and sparser than state-of-the-art online classification algorithms.

show abstract

“…However, it can be approximated arbitrarily well by the optimal mapping in the class by varying the class parameter, and it can be determined only when the complete data stream is observed. 2 In addition to achieving, we might well outperform since our approach is data driven and based on combination of partitions, i.e., we do not rely on a single fixed partition. 3 The convergence rates given here samples our general regret results (after averaging over T ) in the case of BT partitioning.…”

mentioning

confidence: 99%

Asymptotically Optimal Contextual Bandit Algorithm Using Hierarchical Structures

Neyshabouri

Gokcesu

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

We propose an online algorithm for sequential learning in the contextual multiarmed bandit setting. Our approach is to partition the context space and, then, optimally combine all of the possible mappings between the partition regions and the set of bandit arms in a data-driven manner. We show that in our approach, the best mapping is able to approximate the best arm selection policy to any desired degree under mild Lipschitz conditions. Therefore, we design our algorithm based on the optimal adaptive combination and asymptotically achieve the performance of the best mapping as well as the best arm selection policy. This optimality is also guaranteed to hold even in adversarial environments since we do not rely on any statistical assumptions regarding the contexts or the loss of the bandit arms. Moreover, we design an efficient implementation for our algorithm using various hierarchical partitioning structures, such as lexicographical or arbitrary position splitting and binary trees (BTs) (and several other partitioning examples). For instance, in the case of BT partitioning, the computational complexity is only log-linear in the number of regions in the finest partition. In conclusion, we provide significant performance improvements by introducing upper bounds (with respect to the best arm selection policy) that are mathematically proven to vanish in the average loss per round sense at a faster rate compared to the state of the art. Our experimental work extensively covers various scenarios ranging from bandit settings to multiclass classification with real and synthetic data. In these experiments, we show that our algorithm is highly superior to the state-of-the-art techniques while maintaining the introduced mathematical guarantees and a computationally decent scalability.

show abstract

Budget Online Learning Algorithm for Least Squares SVM

Cited by 22 publications

References 24 publications

Toward Budgeted Online Kernel Ridge Regression on Streaming Data

Toward Budgeted Online Kernel Ridge Regression on Streaming Data

A Noise-Resilient Online Learning Algorithm for Scene Classification

Asymptotically Optimal Contextual Bandit Algorithm Using Hierarchical Structures

Contact Info

Product

Resources

About