Evaluating automatically parallelized versions of the support vector machine

Codreanu, Valeriu; Droge, Bob; Williams, David M.; Yasar, Burhan; Yang, Po; Liu, Baoquan; Dong, Feng; Surinta, Olarik; Schomaker, Lambert; Roerdink, Jos B. T. M.; Wiering, Marco A.

doi:10.1002/cpe.3413

Cited by 9 publications

(4 citation statements)

References 46 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their algorithm can be improved in the following two aspects: (i) the level of parallelism is low, since only the kernel value computation is parallelized; (ii) it is not scalable to large datasets due to the assumption that the whole kernel matrix can be stored in the CPU memory. Codreanu et al [7] developed a GPU-based SVM training algorithm that requires holding the training dataset in the GPU memory and uses approximation. In this article, we are interested in improving the efficiency and scalability of the SVM training for regression without approximation.…”

Section: Svm Training Using Gpusmentioning

confidence: 99%

“…We use four real world datasets from the UCI Machine Learning Repository 6 and the LibSVM site 7 . Specifically, the datasets are: (i) Amazon, which contains an anony-mized sample of access provisioned within the Amazon company and person business title serves as the target value of the regression; (ii) CT-slices, which contains features extracted from CT images, and the relative location of the image on the axial axis serves as the target value; (iii) E2006-tfidf, which contains features of corpus and tf-idf of unigrams, and (iv) KDD-CUP'98, which is the dataset used for the KDD Cup 1998 Data Mining Tools Competition, and their first features serve as the target values.…”

Section: Experimental Studymentioning

confidence: 99%

See 1 more Smart Citation

Scalable and fast SVM regression using modern hardware

et al. 2017

View full text Add to dashboard Cite

Support Vector Machine (SVM) regression is an important technique in data mining. The SVM training is expensive and its cost is dominated by: (i) the kernel value computation, and (ii) a search operation which finds extreme training data points for adjusting the regression function in every training iteration. Existing training algorithms for SVM regression are not scalable to large datasets because: (i) each training iteration repeatedly performs expensive kernel value computations, which is inefficient and requires holding the whole training dataset in memory; (ii) the search operation used in each training iteration considers the whole search space which is very expensive.In this article, we significantly improve the scalability and efficiency of SVM regression by exploiting the high performance of Graphics Processing Units (GPUs) and solid state drives (SSDs). Our key ideas are as follows. (i) To reduce the cost of repeated kernel value computations and avoid holding the whole training dataset in the GPU memory, we precompute all the kernel values and store them in the CPU memory extended by the SSD; together with an efficient strategy to read the precomputed kernel values, reusing precomputed kernel values with an efficient retrieval is much faster than computing them on-the-fly. This also alleviates the restriction that the training dataset has to fit into the GPU memory, and hence makes our algorithm scalable to large datasets, especially for large datasets with very high dimensionality. (ii) To enhance the performance of the frequently used search operation,

show abstract

Section: Svm Training Using Gpusmentioning

confidence: 99%

Section: Experimental Studymentioning

confidence: 99%

Scalable and fast SVM regression using modern hardware

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Codreanu et al [23], in their paper Evaluating Automatically Parallelized Versions of the Support Vector Machine, deal with the parallelization on multi-core computers of SVM supervised learning algorithm. They propose a new gradient-ascent-based SVM algorithm combined with a particle swarm optimization algorithm for automatic parameter tuning.…”

Section: Pattern Recognitionsmentioning

confidence: 99%

New advances in High Performance Computing and simulation: parallel and distributed systems, algorithms, and applications

Smari

Bakhouya

Fiore

et al. 2016

Concurrency and Computation

View full text Add to dashboard Cite

INTRODUCTIONRecent developments in research and technological studies have shown that High Performance Computing (HPC) will indeed lead to advances in several areas of Science, engineering, and technology, permitting the successful completion of more computationally intensive and dataintensive problems such as those in healthcare, biomedical and biosciences, climate and environmental changes, multimedia processing, design and manufacturing of advanced materials, geology, astronomy, chemistry, physics, and even financial systems. However, further research is required for developing computing infrastructures, models to support newly evolving architectures, programming paradigms, tools to simulate and evaluate new approaches and solutions, and programming languages that are appropriate for the new and emerging domains and applications.The development of the HPC infrastructure has been accelerated by the advances in silicon technology, which permitted the design of complex systems able to incorporate many hardware and software blocks and cores. More precisely, recent rapid advances in technology and design tools enabled engineers to design systems with hundreds of cores, called multi-processor system-on-chip. These systems are composed of several processing elements, that is, dedicated hardware and software components that are interconnected by an on-chip interconnect. According to Moore's law, the number of cores on-chip will double every 18 months; therefore, thousands of cores-on-chip will be integrated in the next 20 years to meet the power and performance requirements of applications.Moreover, current trends on the road to exascale are moving toward the integration of more and more cores into a single chip [1,2]. For example, accelerators and heterogeneous processing offer some opportunities to greatly increase computational performance and to match increasing application requirements [3]. Engineering these computing systems is one of the most dynamic fields in modern Science and technology. That said, there will continue to be a growing demand for more powerful HPC in the upcoming years, not just to tackle basic mounting computing needs but also to lay out the foundations for the HPC market that is becoming potentially larger than the desktop/laptop computer market. Furthermore, HPC is turning out to be a major source of hope for future applications development that require greater amounts of computing resources in various modern Science domains such as bioengineering, nanotechnology, and energy where HPC capabilities are mandatory in order to run simulations and perform visualization tasks.At the time of writing this editorial, petaflop computing is well established [4]. Several architectures are making major breakthroughs: commodity, accelerators on commodity, and special-purpose cores. All of top 500 systems are based on multicore technologies [5]. HPC usage is growing considerably, especially in industry. And significant efforts toward establishing exascale are underway. At the same time, several challenge...

show abstract

“…ThunderSVM [14] is a rather new SVM implementation supporting CPUs and NVIDIA Graphics Processing Units (GPUs) using the Compute Unified Device Architecture (CUDA). Additionally, there have been various other attempts to port SVMs efficiently to GPUs, e.g., with CUDA [15,16,17], but also using other languages and frameworks such as the Open Computing Language (OpenCL) [18], Python [19,20], OpenACC [21], or SYCL [22]. However, all approaches mentioned so far are based on SMO, an inherently serial algorithm, and, therefore, not very well suited for high performance GPU implementations and vast data sets.…”

Section: Introductionmentioning

confidence: 99%

PLSSVM: A (multi-)GPGPU-accelerated Least Squares Support Vector Machine

Craen¹,

Breyer²,

Pflüger³

2022

Preprint

View full text Add to dashboard Cite

Machine learning algorithms must be able to efficiently cope with massive data sets. Therefore, they have to scale well on any modern system and be able to exploit the computing power of accelerators independent of their vendor. In the field of supervised learning, Support Vector Machines (SVMs) are widely used. However, even modern and optimized implementations such as LIBSVM or ThunderSVM do not scale well for large nontrivial dense data sets on cutting-edge hardware: Most SVM implementations are based on Sequential Minimal Optimization, an optimized though inherent sequential algorithm. Hence, they are not well-suited for highly parallel GPUs. Furthermore, we are not aware of a performance portable implementation that supports CPUs and GPUs from different vendors.We have developed the PLSSVM library to solve both issues. First, we resort to the formulation of the SVM as an least squares problem. Training an SVM then boils down to solving a system of linear equations for which highly parallel algorithms are known. Second, we provide a hardware independent yet efficient implementation: PLSSVM uses different interchangeable backends-OpenMP, CUDA, OpenCL, SYCL-supporting modern hardware from various vendors like NVIDIA, AMD, or Intel on multiple GPUs. PLSSVM can be used as a drop-in replacement for LIBSVM. We observe a speedup on CPUs of up to 10 compared to LIBSVM and on GPUs of up to 14 compared to ThunderSVM. Our implementation scales on many-core CPUs with a parallel speedup of 74.7 on up to 256 CPU threads and on multiple GPUs with a parallel speedup of 3.71 on four GPUs.The code, utility scripts, and the documentation are available on GitHub: https://github.com/SC-SGS/PLSSVM.

show abstract

Evaluating automatically parallelized versions of the support vector machine

Cited by 9 publications

References 46 publications

Scalable and fast SVM regression using modern hardware

Scalable and fast SVM regression using modern hardware

New advances in High Performance Computing and simulation: parallel and distributed systems, algorithms, and applications

PLSSVM: A (multi-)GPGPU-accelerated Least Squares Support Vector Machine

Contact Info

Product

Resources

About