Distributed optimization of multi-class SVMs

Alber, Maximilian; Zimmert, Julian; Doǧan, Ürün; Kloft, Marius

doi:10.1371/journal.pone.0178161

Cited by 12 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SVMs not only perform binary classification, they are also able to do multiclass classification. Four such models are: all-vs-all (AVA) SVM, one-vs-all (OVA) SVM, structured SVM [43], and the Weston and Watkins version [44]. Besides linear classification, SVMs can perform non-linear classification.…”

Section: Combining Classification and Regression Tasks Support Vectormentioning

confidence: 99%

Machine Learning Algorithms for Smart Data Analysis in Internet of Things Environment: Taxonomies and Research Trends

et al. 2020

View full text Add to dashboard Cite

Machine learning techniques will contribution towards making Internet of Things (IoT) symmetric applications among the most significant sources of new data in the future. In this context, network systems are endowed with the capacity to access varieties of experimental symmetric data across a plethora of network devices, study the data information, obtain knowledge, and make informed decisions based on the dataset at its disposal. This study is limited to supervised and unsupervised machine learning (ML) techniques, regarded as the bedrock of the IoT smart data analysis. This study includes reviews and discussions of substantial issues related to supervised and unsupervised machine learning techniques, highlighting the advantages and limitations of each algorithm, and discusses the research trends and recommendations for further study.

show abstract

Section: Combining Classification and Regression Tasks Support Vectormentioning

confidence: 99%

Machine Learning Algorithms for Smart Data Analysis in Internet of Things Environment: Taxonomies and Research Trends

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Existing data-dependent analyses build on either the structural result (1) or (3), which either ignores the correlation among predictors associated to individual class labels or requires f i to be Lipschitz continuous w.r.t. the 2 -norm.…”

Section: Data-dependent Bounds By Gaussian Complexitiesmentioning

confidence: 99%

“…Motivated by the ambiguity in class labels caused by the rapid increase in number of classes in modern computer vision benchmarks, Lapin et al [43,44] introduce the top-k MC-SVM by using the topk hinge loss to allow k predictions for each object x. For any t ∈ R c , let the bracket [·] denote a permutation such that [j] is the index of the j-th largest score, i.e., t [1]…”

Section: Top-k Mc-svmmentioning

confidence: 99%

See 1 more Smart Citation

Data-Dependent Generalization Bounds for Multi-Class Classification

Lei

Doǧan

Kloft

2019

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

In this paper, we study data-dependent generalization error bounds exhibiting a mild dependency on the number of classes, making them suitable for multi-class learning with a large number of label classes. The bounds generally hold for empirical multi-class risk minimization algorithms using an arbitrary norm as regularizer. Key to our analysis are new structural results for multiclass Gaussian complexities and empirical ∞-norm covering numbers, which exploit the Lipschitz continuity of the loss function with respect to the 2-and ∞-norm, respectively. We establish data-dependent error bounds in terms of complexities of a linear function class defined on a finite set induced by training examples, for which we show tight lower and upper bounds. We apply the results to several prominent multi-class learning machines, exhibiting a tighter dependency on the number of classes than the state of the art. For instance, for the multi-class SVM by Crammer and Singer (2002), we obtain a data-dependent bound with a logarithmic dependency which significantly improves the previous square-root dependency. Experimental results are reported to verify the effectiveness of our theoretical findings.

show abstract

“…A easy way is to divide a multiclassification problem into many binary classification problems. Various approaches were proposed to achieve efficient multi classificaton [5,6,7]. The one-aganinst-one (OAO) with a voting strategy [8] and one-against-rest (OAR or one-against-all) with a winer-take-all strategy [9] are two popular methods for doing this dividation.…”

Section: Introductionmentioning

confidence: 99%

Lost-Min Voting Strategies for Speeding up Multi-SVMs

2019

Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering

View full text Add to dashboard Cite

Support vector machines (SVMs) possess good accuracy in big data classification. However, the computational cost in both training and testing stages is a critical issue. The authors recently proposed a twophase sequential minimal optimization to largely reduce the training cost (tested with 3186-70,000-sample datasets). The authors now focus on speeding up the testing speed of SVMs for multi-class classification. A lost-min strategy is proposed to accelerate the voting algorithm used in multi-SVMs. The number of the used binary classifiers is reduced from an order of to (nearly to). The proposed lost-min voting strategy was tested with DNA dataset (bioinformatics), Usps datasets (handwritten digits), Letter dataset (English alphabet) and Satimage dataset (satellite imagery of Earth). The time complexity for all of the datasets approaches to algorithm and the accuracy is remained at the same time.

show abstract

Distributed optimization of multi-class SVMs

Cited by 12 publications

References 34 publications

Machine Learning Algorithms for Smart Data Analysis in Internet of Things Environment: Taxonomies and Research Trends

Machine Learning Algorithms for Smart Data Analysis in Internet of Things Environment: Taxonomies and Research Trends

Data-Dependent Generalization Bounds for Multi-Class Classification

Lost-Min Voting Strategies for Speeding up Multi-SVMs

Contact Info

Product

Resources

About