Ideal kernel tuning: Fast and scalable selection of the radial basis kernel spread for support vector classification

Closed-Form Gaussian Spread Estimation for Small and Large Support Vector Classification

“…The ideal kernel J for a classification problem [22] is a function defined as J (x, y) = 1 when x and y share the class label, and J (x, y) = 0 otherwise. Let {x n } N n=1 be the set of training patterns, c n be the class label of x n , with c n ∈ {1, .…”

“…Here, K nm (σ ) = K (x n , x m , σ ) and the sum is over m > n because K nn = 1 and K nm = K mn , being M = N (N − 1)/2 the number of terms in the sum. Our strategy in previous works [22], [23] was to calculate D(σ ) for several σ values and to select σ minimizing D(σ ). In the following, we develop a method to estimate σ directly from the training set {x n , c n } N n=1 .…”

“…Considering the total execution time T 2 (columns 7 and 8), DGL is faster than LSVM in 10 of 17 datasets where LSVM does not fail, and on average DGL is 3.3 times faster than LSVM, despite that LSVM uses linear kernel and is designed for large datasets. Note also that in the largest datasets, the ratio T 2 (LSVM)/T 2 (DGL) is much higher than 3.3, e.g., 22.4 in baiot. The time T 3 required by file reading is near T 2 for DGL in some datasets, e.g., in record, kddcup, baiot, and kitsune, so the execution of DGL is very fast and most of the time is spent just in data reading.…”

“…A first group of experiments were run over a collection of 30 datasets of small size (Table I), with less than 15 000 patterns. The γ of SVM was estimated: 1) using DGS; 2) using GS, with values in the set = {2 i } 20 i=−20 ; and 3) selecting the value in that minimizes D(γ ) in ( 4), named D-minimization for small datasets (DMS), equivalent to IKT [22]. The regularization λ was always tuned in the set {2 i } 15 i=−7 to maximize kappa on the validation set.…”

“…In a previous work [22], we proposed the ideal kernel tuning (IKT), where σ minimizes the difference between RBF and ideal kernel matrices. On 37 datasets up to 1 million of patterns, IKT achieved lower time (up to 384 s) and memory requirements, with performance similar to GS, KDE, GA, Bayesian search, and PSO.…”