A parallel solver for large quadratic programs in training support vector machines

Zanghirati, Gaetano; Zanni, Luca

doi:10.1016/s0167-8191(03)00021-8

Cited by 120 publications

(70 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The above major contributions summarize the work of how to implement a fast classic SVM in sequential programming. Some earlier works using parallel techniques in SVM can be found in [4,6,23] and [12]. Cao et al presented a very practical Parallel SMO [2] implemented with Message Passing Interface on a cluster system.…”

Section: Background and Related Workmentioning

confidence: 99%

GPUSVM: a comprehensive CUDA based support vector machine package

Salman

Test

et al. 2011

Open Computer Science

View full text Add to dashboard Cite

GPUSVM (Graphic Processing Unit Support Vector Machine) is a Computing Unified Device Architecture (CUDA)based Support Vector Machine (SVM) package. It is designed to offer an end-user a fully functional and user friendly SVM tool which utilizes the power of GPUs. The core package includes an efficient cross validation tool, a fast training tool and a predicting tool. In this article, we first introduce the background theory of how we build our parallel SVM solver using CUDA programming model. Then we compare our GPUSVM package with the popular state of the art Libsvm package on several well known datasets. The preliminary results have shown one to two orders of magnitude speed improvement in both training and predicting phases compared to Libsvm using our Tesla server.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

GPUSVM: a comprehensive CUDA based support vector machine package

Salman

Test

et al. 2011

Open Computer Science

View full text Add to dashboard Cite

show abstract

“…For example, Zanghirati and Zanni [28] proposed a parallel implementation of SVM-light, especially effective for Gaussian kernels; Cao et al [3] also parallelized a slightly modified SMO algorithm. For these two papers, the authors conducted experiments on up to 32 processors with 60k training samples, claiming a speed-up of approximately 20 times.…”

Section: Sequential Minimal Optimization (Smo)mentioning

confidence: 99%

P-packSVM: Parallel Primal grAdient desCent Kernel SVM

Zhu

Chen

Wang

et al. 2009

2009 Ninth IEEE International Conference on Data Mining

View full text Add to dashboard Cite

Abstract-It is an extreme challenge to produce a nonlinear SVM classifier on very large scale data. In this paper we describe a novel P-packSVM algorithm that can solve the Support Vector Machine (SVM) optimization problem with an arbitrary kernel. This algorithm embraces the best known stochastic gradient descent method to optimize the primal objective, and has / dependency in complexity to obtain a solution of optimization error . The algorithm can be highly parallelized with a special packing strategy, and experiences sub-linear speed-up with hundreds of processors. We demonstrate that P-packSVM achieves accuracy sufficiently close to that of SVM-light, and overwhelms the state-of-the-art parallel SVM trainer PSVM in both accuracy and efficiency. As an illustration, our algorithm trains CCAT dataset with 800k samples in 13 minutes and 95% accuracy, while PSVM needs 5 hours but only has 92% accuracy. We at last demonstrate the capability of P-packSVM on 8 million training samples.

show abstract

“…Unlike sequential or centralized approaches in the literature [2], [4]- [7], the focus here is exclusively on parallel update schemes for machine learning allowing individual processing units to do simultaneous computations. Most of the existing contributions investigate training SVMs either sequentially or in parallel first and then fusing them into a centralized classifier.…”

Section: Introductionmentioning

confidence: 99%

A distributed machine learning framework

Alpcan

Bauckhage

2009

Proceedings of the 48h IEEE Conference on Decision and Control (CDC) Held Jointly With 2009 28th Chinese Control Conference

View full text Add to dashboard Cite

Abstract-A distributed online learning framework for support vector machines (SVMs) is presented and analyzed. First, the generic binary classification problem is decomposed into multiple relaxed subproblems. Then, each of them is solved iteratively through parallel update algorithms with minimal communication overhead. This computation can be performed by individual processing units, such as separate computers or processor cores, in parallel and possibly having access to only a subset of the data.Convergence properties of continuous-and discrete-time variants of the proposed parallel update schemes are studied. A sufficient condition is derived under which synchronous and asynchronous gradient algorithms converge to the approximate solution. Subsequently, a class of stochastic update algorithms, which may arise due to distortions in the information flow between units, is shown to be globally stable under similar sufficient conditions. Active set methods are utilized to decrease communication and computational overhead. A numerical example comparing centralized and distributed learning schemes indicates favorable properties of the proposed framework such as configurability and fast convergence.

show abstract

A parallel solver for large quadratic programs in training support vector machines

Cited by 120 publications

References 14 publications

GPUSVM: a comprehensive CUDA based support vector machine package

GPUSVM: a comprehensive CUDA based support vector machine package

P-packSVM: Parallel Primal grAdient desCent Kernel SVM

A distributed machine learning framework

Contact Info

Product

Resources

About