Combining meta-learning and search techniques to select parameters for support vector machines

Gomes, Taciana A. F.; Prudêncio, Ricardo B. C.; Soares, Carlos; Rossi, André Luis Debiaso; Carvalho, André Carlos Ponce de Leon Ferreira

doi:10.1016/j.neucom.2011.07.005

Cited by 110 publications

(77 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to some studies, the hyper-parameters of SVMs should always be tuned when looking for the best predictive performance [14], [25], [27]. Conversely, other studies reported experimental results where the default values provided pre dictive performances similar to those obtained by optimized hyper-parameters [3].…”

Section: Resultsmentioning

confidence: 99%

“…Other studies combined MTL with optimization techniques for the selection of hyper-parameter values [25]- [28]. In these studies, MTL recommends hyper-parameter values for the initial population of a search technique, leading optimization methods to a faster convergence.…”

Section: Related Workmentioning

confidence: 99%

“…• one with 17 STATLOG meta-features, originally used in the STATLOG project [35], and later in many similar studies [22], [25], [26]. These meta-features are simple measures based on statistics, such as the number of classes, number of attributes, class distri bution and so on;…”

Section: Meta-featuresmentioning

confidence: 99%

See 2 more Smart Citations

To tune or not to tune: Recommending when to adjust SVM hyper-parameters via meta-learning

Mantovani

Rossi

Vanschoren

et al. 2015

2015 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Abstract-Many classification algorithms, such as Neural Net works and Support Vector Machines, have a range of hyper parameters that may strongly affect the predictive performance of the models induced by them. Hence, it is recommended to define the values of these hyper-parameters using optimization techniques. While these techniques usually converge to a good set of values, they typically have a high computational cost, because many candidate sets of values are evaluated during the optimization process. It is often not clear whether this will result in parameter settings that are significantly better than the default settings. When training time is limited, it may help to know when these parameters should definitely be tuned. In this study, we use meta-learning to predict when optimization techniques are expected to lead to models whose predictive performance is better than those obtained by using default parameter settings. Hence, we can choose to employ optimization techniques only when they are expected to improve performance, thus reducing the overall computational cost. We evaluate these meta-learning techniques on more than one hundred data sets. The experimental results show that it is possible to accurately predict when optimization techniques should be used instead of default values suggested by some machine learning libraries.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

To tune or not to tune: Recommending when to adjust SVM hyper-parameters via meta-learning

Mantovani

Rossi

Vanschoren

et al. 2015

2015 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…In this paper, we introduced the kernel SVMs (KSVMs), which extends original linear SVMs to nonlinear SVM classifiers by applying the kernel function to replace the dot product form in the original SVMs [20]. The KSVMs allow us to fit the maximum-margin hyperplane in a transformed feature space.…”

Section: Introductionmentioning

confidence: 99%

An MR Brain Images Classifier via Principal Component Analysis and Kernel Support Vector Machine

Zhang¹,

Wu²

2012

PIER

251

146

View full text Add to dashboard Cite

Abstract-Automated and accurate classification of MR brain images is extremely important for medical analysis and interpretation. Over the last decade numerous methods have already been proposed. In this paper, we presented a novel method to classify a given MR brain image as normal or abnormal. The proposed method first employed wavelet transform to extract features from images, followed by applying principle component analysis (PCA) to reduce the dimensions of features. The reduced features were submitted to a kernel support vector machine (KSVM). The strategy of Kfold stratified cross validation was used to enhance generalization of KSVM. We chose seven common brain diseases (glioma, meningioma, Alzheimer's disease, Alzheimer's disease plus visual agnosia, Pick's disease, sarcoma, and Huntington's disease) as abnormal brains, and collected 160 MR brain images (20 normal and 140 abnormal) from Harvard Medical School website. We performed our proposed methods with four different kernels, and found that the GRB kernel achieves the highest classification accuracy as 99.38%. The LIN, HPOL, and IPOL kernel achieves 95%, 96.88%, and 98.12%, respectively. We also compared our method to those from literatures in the last decade, and the results showed our DWT+PCA+KSVM with GRB kernel still achieved the best accurate classification results. The averaged processing time for a 256 × 256 size image on a laptop of P4 IBM with 3 GHz processor and 2 GB RAM is 0.0448 s. From the experimental data, our method was effective and rapid. It could be applied to the field of MR brain image classification and can assist the doctors to diagnose where a patient is normal or abnormal to certain degrees.

show abstract

“…The most commonly used variants are the maximum margin L 1 norm SVM [1], and the least squares SVM (LSSVM) [2], both of which require the solution of a quadratic programming problem. In the last few years, SVMs have been applied to a number of applications to obtain cutting edge performance; novel uses have also been devised, where their utility has been amply demonstrated [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24]. SVMs were motivated by the celebrated work of Vapnik and his colleagues on generalization, and the complexity of learning.…”

Section: Introductionmentioning

confidence: 99%

Learning a hyperplane classifier by minimizing an exact bound on the VC dimension1

Jayadeva

2015

Neurocomputing

View full text Add to dashboard Cite

The VC dimension measures the complexity of a learning machine, and a low VC dimension leads to good generalization. While SVMs produce state-of-the-art learning performance, it is well known that the VC dimension of a SVM can be unbounded; despite good results in practice, there is no guarantee of good generalization. In this paper, we show how to learn a hyperplane classifier by minimizing an exact, or Θ bound on its VC dimension. The proposed approach, termed as the Minimal Complexity Machine (MCM), involves solving a simple linear programming problem. Experimental results show, that on a number of benchmark datasets, the proposed approach learns classifiers with error rates much less than conventional SVMs, while often using fewer support vectors. On many benchmark datasets, the number of support vectors is less than one-tenth the number used by SVMs, indicating that the MCM does indeed learn simpler representations.

show abstract

Combining meta-learning and search techniques to select parameters for support vector machines

Cited by 110 publications

References 18 publications

To tune or not to tune: Recommending when to adjust SVM hyper-parameters via meta-learning

To tune or not to tune: Recommending when to adjust SVM hyper-parameters via meta-learning

An MR Brain Images Classifier via Principal Component Analysis and Kernel Support Vector Machine

Learning a hyperplane classifier by minimizing an exact bound on the VC dimension1

Contact Info

Product

Resources

About